BITP 3213

Software Testing & Quality Assurance

by Madam Nor Haslinda Hj Ismail Email: lynda@utem.edu.my Phone: 331 6521 Room: Ground Floor FTMK building
Semester 1, 2010/2011 Fakulti Teknologi Maklumat dan Komunikasi

Week 1 Introduction to Software Testing

Fakulti Teknologi Maklumat dan Komunikasi

can lead to

can lead to

Human Error

Fault

Failure

Fakulti Teknologi Maklumat dan Komunikasi

Overview

What is Software Testing? Objectives of Software Testing General Failure Terminology Impact of Software Error Real-life cases Critical Choice in Software Testing Fundamental of Software Process Type of Software Testing Testing and Quality Career in Software Testing
Fakulti Teknologi Maklumat dan Komunikasi

Software Testing Overview

Studies have observed that the cost of correcting software bugs has tremendously increased the total cost of a software project. Defect. Fault. Problem. Error. Incident. Variance. Failure. Inconsistency It is crucial to detect the bugs at an early stage.

Software Testing
Fakulti Teknologi Maklumat dan Komunikasi

When to consider that a software has Bugs? (The 5 rules)

1. 2.

3.

4.

5.

The software doesnt do something that the product specification says. The software does something that the product specification says it shouldnt do. The software does something that the product specification doesnt mention The software doesnt do that the product specification doesnt mention but should The software is difficult to understand, hard to use, oranything which a tester think that will be viewed as not right by the end user.

Fakulti Teknologi Maklumat dan Komunikasi

The 5 Rules : Rule 1

The software doesnt do something that the product specification says. Example :-

The + key of a calculator should do the addition. But it doesnt do any calculation or instead it works as subtraction.

Fakulti Teknologi Maklumat dan Komunikasi

The 5 Rules : Rule 2

The software does something that the product specification says it shouldnt do. Example:-

The product specification specifies that the calculator should never crash, lock up or freeze. Yet, when you press Enter key, it stops responding to any of your next input.

Fakulti Teknologi Maklumat dan Komunikasi

The 5 Rules : Rule 3

The software does something that the product specification doesnt mention
Example:It is specified that the calculator should only do addition, subtraction, multiplication and division. Nevertheless, there is an additional of a square root function, which being added by programmer because he/she things the function is nice to have.
Fakulti Teknologi Maklumat dan Komunikasi

The 5 Rules : Rule 4

The software doesnt do that the product specification doesnt mention but should Example:The calculation of a calculator is wrongly calculated if the battery is weak. This may not be specified in the specification.

Fakulti Teknologi Maklumat dan Komunikasi

The 5 Rules : Rule 5

The software is difficult to understand, hard to use, or- anything which a tester think that will be viewed as not right by the end user.
Example:As a tester, you will be the first person to use the software product. Thus, if the tester feels there is something not right for whatever reason, then its consider as a bug.
Fakulti Teknologi Maklumat dan Komunikasi

Why bugs occur in Software Development?

Missing requirements or the requirements/specifications are not well documented. e.g. Contradiction, ambiguity, incomplete requirement Volatile requirements The requirements are constantly changing. Communication problem among team members. No team work

Fakulti Teknologi Maklumat dan Komunikasi

What is Software Testing? Definition

Process of exercising software to confirm that it satisfies requirements and identify differences between expected and actual results (Khanapi, 2004)
Testing is the process of executing a program with intention of finding errors (Myers, 1979)
Fakulti Teknologi Maklumat dan Komunikasi

Definition
The measurement of software quality
(Hetzel, 1983)

The process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component (IEEE, 1990)
Fakulti Teknologi Maklumat dan Komunikasi

Definition : Summary

Software testing is a process by which people, methods, measurements, tools and equipment are integrated to test a software product.

Fakulti Teknologi Maklumat dan Komunikasi

Process A quality process and planning determines the success of the test effort People Software testing is a professional discipline , thus need trained and skilled people with the right and positive attitudes. Methods There are different methods and techniques to be applied in testing. E.g White-box, black box,
Fakulti Teknologi Maklumat dan Komunikasi

Measurements - Used to calculate the quality of a software product. Tools and Equipments
-

A wide variety of tools to make the testing more efficient and faster. E.g. Using automated tool to re-run the test for a modified software product. Save time and able to test thoroughly.

Fakulti Teknologi Maklumat dan Komunikasi

Basic forms of Testing

Verification Are we building the product right? Is the process of evaluating a system or component to determine whether the products of a given development phase satisfy the condition imposed at the start of the phase. Validation Are we building the right product Is the process of evaluating a system or component during or at the end of the development process to determine whether it satisfied specified requirements.
Fakulti Teknologi Maklumat dan Komunikasi

Objectives of Software Testing

To identify and reveal as many errors as possible in the tested software To produce a quality software product, which the software can be used, and used successfully

To bring the tested software, after correction of the identified errors and retesting, to an acceptable level of quality

Fakulti Teknologi Maklumat dan Komunikasi

Objectives of Software Testing

To perform the required tests efficiently and effectively, within budgetary and scheduling limitations To compile a record of software errors for use in error prevention

Fakulti Teknologi Maklumat dan Komunikasi

General Failure Terminology

Error :

An incorrect implementation of a program

An incorrect or unexpected behavior of a program in response to a real-world condition resulting from errors

Fault :

Failure :

Occurs when a program is prevented from achieving its mission resulting from faults
Fakulti Teknologi Maklumat dan Komunikasi

Relationship : Error-Fault-Failure
Error made by a person. - Avoidable through training, process improvement, or similar

Leading to a fault in the program - Detectable in review

Causing a noticeable failure in the system - Detectable trough testing
Fakulti Teknologi Maklumat dan Komunikasi

Impact of software error

Case 1 The first Venus probe missed its target in 1979 because a full stop(.) was used in place of comma(,) in a Fotran program. The company lost several hundred million dollars worth of effort.

Fakulti Teknologi Maklumat dan Komunikasi

Impact of software error

Case 2 In 1984, the southern French Tarn Valley was flooded due to the failure of an automatic lock system at Requista where it couldnt recognize the faulty flood message. A massive flood which caused catastrophic.

Fakulti Teknologi Maklumat dan Komunikasi

Impact of software error

Case 3 Years before year 2000, most systems were using the 2-digit date format. (E.g. 1973 as 73). When it hits year 2000, problems occurred when the systems recognized the 00 as year 1900 instead of 2000. Estimated that several hundred billions were spent, worldwide to correct the date format.
Fakulti Teknologi Maklumat dan Komunikasi

Critical choices in Software Testing What, when and how to test?

What to test?
Test the right things! ensure the most critical things are tested and not waste limited testing resources on less important items. When to test? Test early! focus on detecting the error closer to the phase where it introduce. E.g. V model Testing
Fakulti Teknologi Maklumat dan Komunikasi

How to test? Test effectively and cost-effectively! Identify the critical/important area to be tested by considering the worthwhile from financial point of view.
The more effective the error detection, the greater the savings in development and maintenance costs over the life of the product. Several examples have indicated that partial testing can yield saving of 1.5 times its cost; full testing can yields savings up to 2 times its cost (Lewis, 1992: p.280)
Fakulti Teknologi Maklumat dan Komunikasi

RISKS and Prioritization

To determine :

What to test first? What to test most? How thoroughly to test each item?
risk of missing important faults risk of incurring failure costs risk of releasing untested or under-tested software risk of losing credibility and market share risk of missing a market window risk of over-testing, ineffective testing

Example of general risks to be considered :

Fakulti Teknologi Maklumat dan Komunikasi

Fundamental of Test Process

Test planning and control Test analysis and test design Test implementation and test execution Test evaluation of the test exit criteria

Post testing activities : Review

Fakulti Teknologi Maklumat dan Komunikasi

Types of Software Testing

Unit Testing Test particular function or code module : micro level testing Integration Testing Test the integrated/combined components of code modules, individual application, client and server application and etc. Performance Testing Test the performance of a software product System Testing Test the system as a whole.
Fakulti Teknologi Maklumat dan Komunikasi

Regression Testing Is used to check that the changes made to the program have not introduce any new faults in the system User Acceptance Testing Test the final end product by a group of end users to determine the satisfaction of the customers Usability Testing Used to test user-friendliness of the software product

Fakulti Teknologi Maklumat dan Komunikasi

What makes a good Tester?

They are explorers As a tester, you are not bound to only one specific application/product. You must not afraid to venture into unknown situations. They are troubleshooters You must be able to figure out the reason why something is not working.

Fakulti Teknologi Maklumat dan Komunikasi

What makes a good Tester?

They are relentless Persistence and a strong sense of mission to systematically locate the weaknesses in a complex structure and demonstrates it failures. There are bugs that quickly vanish or difficult to re-create. Rather than dismiss it as fluke, they will try every possible way to find it. They are creative Imaginative. Able to come out with different possible ways/scenarios/approaches in finding the bugs.
Fakulti Teknologi Maklumat dan Komunikasi

What makes a good Tester?

They exercise good judgment - A tester need to make fine decision about what they will test, how long will it take, and whether the problem theyre looking at is really a bug.

They are tactful and diplomatic A tester is always a bearers of bad news. He/she has to convey the bugs to appropriate party. This must be done in professional and diplomatic manner so that, the other party will not feel offended or personally being attacked. A team player attitude.
Fakulti Teknologi Maklumat dan Komunikasi

Testing and Quality

Testing measures the quality of software product. Testing can find faults; when they are removed, software quality (and possibly reliability) is improved.

Fakulti Teknologi Maklumat dan Komunikasi

Career in Software Testing

Why choose career in Software Testing? Career Path /Options
Test Engineer Test > Test Manager

Key areas to move ahead

Increase productivity

Domain knowledge

Wide range of career option

QA Lead QA Manager

Tools Knowledge Automated Testing Certifications E.g. ISTQB, QAI, ASQ

Option by specialized in testing tool

Fakulti Teknologi Maklumat dan Komunikasi

Find the solution!

How much more does it cost to fix a bug found after the product is released that it does from the very start of the project? Give three reason why the product specification is usually the largest source of bugs in a software product Find 2 article from ACM & IEEE Journal regarding Software Testing from Year 2009 until 2011.
Fakulti Teknologi Maklumat dan Komunikasi

Next Chapter

Week 2: Software Testing & SDLC

Fakulti Teknologi Maklumat dan Komunikasi