Seminar Report

Component-Based White-Box Testing

By

Prathab K
Roll No: 06329902

Under the guidance of Prof. Deepak

B. Phatak

KReSIT, Indian Institute of Technology, Bombay.

Mumbai. March, 2007

Table of Contents
1.0 2.0 2.1 2.2 2.3 2.4 3.0 3.1 4.0 4.1 4.2 4.3 4.4 4.5 INTRODUCTION.......................................................................................................................... 3 COMPONENT-BASED SOFTWARE DEVELOPMENT......................................................... 4 WHAT IS A SOFTWARE COMPONENT?............................................................................................ 4 PROPERTIES OF SOFTWARE COMPONENTS IN CBSD ..................................................................... 4 THE ENGINEERING PROCESS FOR SOFTWARE COMPONENTS .......................................................... 5 COMPONENT-BASED SOFTWARE VERSUS TRADITIONAL PROGRAMS .............................................. 6 SOFTWARE COMPONENT TESTING..................................................................................... 7 METHODS TO INCREASE COMPONENT TESTABILITY ...................................................................... 8 WHITE-BOX TESTING METHODS FOR SOFTWARE COMPONENTS ........................... 9 FLOW GRAPH NOTATION ............................................................................................................... 9 PATH TESTING ............................................................................................................................ 10 DATA-FLOW TESTING ................................................................................................................. 11 OBJECT-ORIENTED TESTING ....................................................................................................... 11 CHALLENGES AND ISSUES IN TESTING SOFTWARE COMPONENTS ................................................ 12

5.0 SYSTEMATIC PROCEDURE FOR WHITE-BOX TESTING OF SOFTWARE COMPONENTS ......................................................................................................................................... 12 5.1 5.2 5.3 5.4 6.0 7.0 8.0 WHITE-BOX TESTING TOOLS ....................................................................................................... 13 CONTINUOUS TESTING ............................................................................................................... 13 REGRESSION MODEL OF WHITE-BOX TESTING FOR SOFTWARE COMPONENTS .............................. 14 MAPPING OF TEST CASES TO DEVELOPMENT CODE ..................................................................... 14 CONCLUSIONS .......................................................................................................................... 15 ACKNOWLEDGEMENTS......................................................................................................... 16 REFERENCES............................................................................................................................. 16

Component-Based White-Box Testing

Abstract
In component-based software engineering, software system quality depends on the quality of the components. With the increasing software complexity and reducing development time, white-box testing plays an important role in developing reliable components. Though the availability of white-box testing tools makes the process easier, it does not provide a systematic and effective white-box testing process. Especially, the component that undergoes frequent modifications, demands a systematic regression model of white-box testing. This seminar report discusses on the white-box testing methods for software components and focuses on the systematic regression model for white-box testing. And how to reduce the development time by the process of continuous testing at unit level and find out the reusable test cases in a component's test suite to support its evolution.

1.0 Introduction
With the increasing complexity of software applications and reducing development time, the software quality and reliability have become an important concern of the software industry. To address the software quality, its development time and cost, a ComponentBased Software Development (CBSD) is best approach to be adopted. Component-Based Software Engineering (CBSE) is best practice to reduce complexity of the process. It is primarily concerned with developing software from pre-produced parts and the ability to reuse those parts in other contexts. Its major objective is to reduce the software development cost and time by reusing available components, including third-party and in-house grown components. CBSD demands high quality components to achieve its goals, which requires effective testing in place to provide the quality assurance of the components. Testing accounts for significant part of the software development lifecycle. White-box testing plays a major role in the total testing efforts. Studies by Barry Boehm reveal that coding and unit test takes 36 to 42% of the total resources consumed in software projects of various sizes [1]. In component-based development with demand of high quality and reliability components, white-box testing is a key phase in the testing process. In CBSD, the components undergo frequent changes during development or after the development due to the change in the requirements and other platform (hardware and software) compatibility issues. The real challenge comes in re-testing the modified components at unit level and there by reducing the regression testing time. This demands a systematic method of white-box testing of the components which undergo frequent changes and proper software change analysis and test suite selection to assist in the quality assurance of the component. This report focuses on the white-box testing for software components and how to have a systematic approach for component-based white-box testing. This report is structured as follows. Next section discusses on the component-based software development and its importance. Third section focuses on the software component testing and methods to increase the component testability. Section 4 investigates the white-box testing methods for software components and the issues involved. Section 5 discusses the role of tools in

white-box testing and with the emphasis to have a systematic procedure for white-box testing and the re-test model for the white-box testing activities. Section 6 provides the conclusion of this study of having tight mapping of test cases to source code for an effective white-box testing.

2.0 Component-based Software Development

To cope with the software complexity in software application systems, forced software vendors to look for cost-effective methods to construct complicated software systems in a short development life cycle to meet marketing needs. Consequently, there is a growing demand of searching for a new, efficient, and cost-effective software development paradigm. One of the most promising solutions today is the component-based software development approach. This approach is based on the idea that software systems can be developed by selecting appropriate off-the-shelf components and then assembling them with well-defined software architecture [2]. Component-based software development (CBSD) can significantly reduce development cost and time-to-market, and improve maintainability, reliability and overall quality of software systems.

2.1

What is a software component?

What is a software component? One of the earliest definitions is given by Gready Booch [3]: A reusable software component is a logically cohesive, loosely coupled module that denotes a single abstraction. Though several definitions for software component exists, Bill Councill and George T. Heineman gave a new definition to emphasize the importance of a consistent component model and its composition standard in building components and component-based software [4]: A software component is a software element that confirms to a component model and can be independently deployed and composed without modification according to a composition standard. In simpler terms, the notion of the software components can be classified into three types [5]: A reusable module is an independent and deliverable software part that encapsulates a functional specification and implementation for reuse by a third party. A reusable component is an independent, deployable, and replaceable software unit that is reusable by a third party based on the unit's specification, implementation, and well-defined contracted interfaces. A composite building block is a reusable component that is developed as a building part to conform a well-defined component model and the accompanying composition standard

2.2

Properties of software components in CBSD

Component properties refer to the essential characteristics of software components. A software component in CBSD must have the following basic properties [5]. Identity: Each component must be uniquely identifiable in its development environment and targeted deployment environment. 4

Modularity and encapsulation: Software components result from partitioning of a software system by focusing on system modularity. Each component encapsulates a set of closely related data elements and implements coherent functional logic to perform a specific task. Independent delivery: Software components must be delivered as independent parts that can be replaced under certain conditions. Contract-based interfaces: An interface between software components defines a contract between the client of an interface and a provider of an implementation for the interface Reusability: The reusable elements of a software component include its analysis specification, component design and design patterns, source code, and executables.

2.3

The engineering process for software components

In CBSD, the notion of building a system by writing code has been replaced with building a system by assembling and integrating existing software components. In contrast to traditional development, where system integration is often the tail end of an implementation effort, component integration is the centerpiece of the approach; thus, implementation has given way to integration as the focus of system construction. Because of this, integrability is a key consideration in the decision whether to acquire, reuse, or build the components. As shown in Figure 1, four major activities characterize the component-based development approach [6]:

Figure 1: Activities of the Component-Based Development Approach

Component qualification: Component qualification is a process of determining "fitness for use of previously-developed components that are being applied in a new system context. When a marketplace of competing products exists, it also involves the selection of the most suitable component. Component adaptation: Individual components are written to meet different requirements, each one making certain assumptions about the context in which it is

deployed. The purpose of adaptation is to ensure that conflicts among components are minimized. Assembling components into systems: Assembly is the integration of components through some well defined infrastructure, which provides the binding that forms a system from disparate components. COTS (Commercial off-the-shelf) components, for example, are usually written to some component model defined by e.g., Enterprise JavaBeans, COM, CORBA, or, more recently, .NET. System evolution: In component-based systems components are the units of change, system evolution is based around the replacing of outdated components by new ones, or, at least, ideally. The treatment of components as plug-replaceable units is a simplistic view of system evolution. In practice, replacing a component may be a non-trivial task, especially when there is a mismatch between the new component and old one, triggering another stage of adaptation with the new component.

2.4

Component-based software versus traditional programs

The main objective of traditional systems is to deliver an efficient and reliable software system. Reusability, maintainability and evolvability are characteristics that are not among the top priorities. Nevertheless, as software systems get larger and larger, and more and more complex, these characteristics become more critical. Component-based systems must properly address these issues in order to achieve the following objectives: shorter time to market, lower cost, better reusability, higher reliability, and maintainability [5]. Below table summarizes the major differences between the component-based software and traditional programs [5]: Perspectives Objective Component-based Software To efficiently develop reliable, maintainable, reusable software systems Component-based software engineering processincludes many unique activities such as component selection, customization, composition Relatively easy to achieve, because the adopted component model and component architectures will reconcile many incompatibility and interoperability issues Reusability can occur at different level: component, component architecture Traditional Software To develop efficient, reliable software systems Traditional software engineering processincludes requirement analysis, design, coding, and testing Difficult to achieve as the entire engineering process is often for one specific environment

Process

Portability

Reusability

Difficult to reuse; for some OO systems, the reuse is limited to individual classes

Perspectives Scalability

Component-based Software Easy to scale: the scalability can be automatically managed by component model

Traditional Software Hard to scale as any change in the code may require changes to all relevant modules or subsystems

Table 1: Component-based software versus Traditional programs

3.0 Software component testing

The system quality depends on the component quality, any defective component causes a ripple impact on all systems built on it. Hence, component validation and quality control is critical to both component vendors and users. Testing Component-based software involves two major parts: testing the individual component to ensure its quality testing the system as whole, which is composed of these components. (Basically, the interaction among the components.) The adequate testing for component-based software necessary because, Inconsistent infrastructure and environment: Heterogeneity is one key feature of component-based software. Heterogeneity may introduce incompatibilities among different programming languages, operating systems, databases, and external operational environments that are used to develop software components or support the execution of software components. Inconsistent interaction model: Software components are developed by different groups of people, who may make different assumptions of how components interact and what details are involved in their interactions. Moreover, a component may expose multiple interfaces, which may have varied constraints and different types of relationships with each other.

The difficulties in adequate testing for component-based software are [5]: Code availability: The lack of source code causes a lot of problems. First of all, all whitebox based testing and maintenance techniques are unavailable. Only black-box based techniques can be adopted. Performance and reliability analysis: Additional overheads are introduced in converting to and from standardized messages during the communication and end up with a degradation of performance. The reliability is an issue since the component is developed in one environment and executed in another environment without testing. Adequacy: Without source code availability of the component it is difficult to check test adequacy.

3.1

Methods to increase component testability

Component testability is two-fold: 1. It refers to the degree to which a component is constructed to facilitate the establishment of component test criteria and the performance of tests to determine whether those criteria have been met. 2. It refers to the degree to which testable and measurable component requirements are clearly given to allow the establishment of test criteria and performance of tests. The following are the approaches to increase software component testability: 1. Framework-based testing facility [9]: In this approach, a well-defined framework (such as a class library) is developed to allow engineers to add program test-support code into components according to the provided application interface of a component test framework. 2. Built-in tests [10]: This approach requires component developers to add built-in tests inside a software component to support self-checking and self-testing. The component tests are built in inside components and the engineers can perform component tests without any external support from a component testing bed and test repository. 3. Systematic component wrapping for testing [8]: This approach uses a systematic way to convert a software component into a testable component by wrapping it with the program code that facilitates software testing. J. Gao et al. proposed a new concept, called testable beans, to enhance component testability. A testable bean is a software component designed to facilitate component testing which includes its functional code that supports the required functional features and data, and also consists of the parts supporting component testing. Below table provides the comparison of the above mentioned approaches [5]: Different Perspectives Programming overhead Testing code separated from source code Software tests inside components Test change impact on components Software change impact on component test FrameworkBased Testing Facility Low No No No No Built-In Tests High No Yes High Yes Systematic Component Wrapping for Testing Very Low Yes No No No

Different Perspectives Component complexity Usage flexibility

FrameworkBased Testing Facility Low High

Built-In Tests Very High Low

Systematic Component Wrapping for Testing High Low

Table 2: Comparison of different component testing facilities

4.0 White-box testing methods for software components

According to IEEE standards [11], white-box testing, also known as structural testing or glass-box testing, is testing that takes into account the internal mechanism of a system or component. White-box testing contrasts with black-box testing in that it requires an internal knowledge of a component. An immediate question arises on the necessity of white-box testing, when black-box testing ensures that requirements have been met? To answer this question, in many program faults can be overlooked by black-box testing. For instance, logic errors, which can account for as many as 32% of all possible faults [12], may not be identified effectively by black-box testing; white-box testing is more suitable for that scenario. Black-box testing is testing against the specification and will discover faults of omission, indicating that part of the specification has not been fulfilled. White box testing is testing against the implementation and will discover faults of commission, indicating that part of the implementation is faulty [13]. There are few shortcomings of white-box testing. It creates a false sense of stability and quality. An application can often pass all unit tests and still appear unavailable to users due to a non-functioning component (a database, connection pool, etc.). This is a critical problem, since it makes the system application team look incompetent. Also inability to validate use-case completeness because they use a different interface to the application. White Box tests focus on pleasing the wrong crowd - developers. The right crowd to please in the real world is application users [14]. But, white-box testing plays an important role in the entire testing effort, by the early detection of core logical faults and there by saving the software cost. The context of white-box testing for software components does not differ much with respect to traditional software systems. But there are some additional issues need to be addressed when applying white-box testing to software components [5]. Lets see about the white-box testing and the different test-adequacy criteria.

4.1

Flow graph notation

White-box-testing techniques rely on the structure of a program. To represent the structure of a program, the flow graph notation is used. A flow graph G = (V, E) is a direct graph, where V is a set of nodes and each node v V represents a basic block. And other terminologies in the flow graph are:

A basic block is a maximal sequence of statements, which satisfy all-or-nothing criterion, which is represented as node in the graph. A decision node is a node with two or more outgoing edges that is used to represent branches in a program. A junction node is a node with two or more incoming edges. Edges represent control moving from one node to another node.

Flow graphs are used to depict logical control flowing of a program. Below is an example of a flow graph.

Figure 2: A flow graph example

4.2

Path testing

Basis path testing [15] is a means for ensuring that all independent paths through a code module have been tested. An independent path is any path through the code that introduces at least one new set of processing statements or a new condition. Basis path testing provides a minimum, lower-bound on the number of test cases that need to be written. Lets look at some the coverage factors used to achieve the path testing [5]. Statement coverage: Every statement of the program is expected to run atleast once. So, in the flow graph all the nodes should be covered atleast once. Branch coverage: In flow graphs, decision nodes are special nodes that will determine the next move of the program. In other words, we need to examine all possible transitions from a decision node. Multiple-condition coverage: Branch-coverage criteria reinforce the testing to cover all outgoing edges of a decision node and consider the predicate as one unit. Multiple condition coverage reports whether every possible combination of boolean subexpressions occurs. As with condition coverage, the sub-expressions are separated by

10

logical-and and logical-or, when present. The test cases required for full multiple condition coverage of a condition are given by the logical operator truth table for the condition. Path coverage: To adopt path-coverage criteria in a practical way, we need to select a representative subset of all paths to retest. The selection needs to remove redundant information in a path. A path with no repeat occurrences of any node and edge will ensure the path coverage. Loop Coverage: This measure reports whether you executed each loop body zero times, exactly once, and more than once (consecutively). For do-while loops, loop coverage reports whether you executed the body exactly once, and more than once. Any program with a backward branch potentially has an infinite number of paths. McCabe's cyclomatic measurement [15] is an example of specifying dependence relationships between paths. Cyclomatic numbering is based on a theorem of graph theory that for any flow graph, there is a set of paths where all other paths can be represented by linear combination of them. Within the set, no paths can be expressed by linear combinations of other paths, and these paths are called independent paths. The cyclomatic number V(G) of a graph G with n vertices, e edges, and p connected components is v(G) = e - n + p Based on the cyclomatic number, a test suite T satisfies the cyclomatic adequacy criterion if and only if all independent paths are executed at least once by the test cases in T.

4.3

Data-flow testing

Data flow testing is a testing technique based on the observation that values associated with variables can effect program execution [16][17]. Data flow testing not only explores program control flows but also pays attention to how a variable is defined and used at different places along control flows, which could lead to more efficient and targeted test suites than pure control-flow-based test suites. There are two types of occurrences of variables in a program, namely definition occurrences and use occurrences. A definition occurrence of a variable is where a value is assigned to a variable. A use occurrence of a variable is where the value of a variable is used in the program [18]. All definitions criterion is a test coverage criterion that requires that an adequate test set should cover all definition occurrences in the sense that, for each definition occurrence, the testing paths should cover a path through which the definition reaches a use of the definition. All use criterion requires that all uses of a definition should be covered. Clearly, all-uses criterion is stronger than all-definitions criterion. There by data-flow testing looks at the lifecycle of a particular piece of data i.e. a variable in an application.

4.4

Object-oriented testing

Nowadays, more and more components are developed by object-oriented(OO) programming languages, which demands the Object-oriented testing is needed to the OO features in the components. Below are essential features needs to be tested:

11

Inheritance: Need to test the inherited methods in the subclass. And Check for the type safety of overriding and overloading methods. Polymorphism: With a statement, an object may be bound to different classes during the run time, and the binding can usually only be determined dynamically. So the contracts of types need to be tested. Binding coverage: To ensure that each object must be exercised at least once in the defined test suite. State-based testing: To test the features like encapsulation, state-based testing will help to analyze the different abstract states that a class can take.

4.5

Challenges and issues in testing software components

There are two major challenges when using white-box-testing techniques to test software components [5]: 1. Software components are usually designed and developed in an isolated environment. When deployed later in another software system, software components may need to be configured or customized in a different environment. Therefore, when testing software components, we can expect to encounter more uncertainties than with traditional software systems. 2. The quality requirements for software components are generally higher than the requirements for traditional software modules. This is because white-box testing is often used at unit level. Faults in a program module may have more chances to be identified in a later stage and can be easily fixed internally. For software components, if faults are identified after the delivery, it is harder and more expensive to fix the faults. Moreover, faults in the software component can also hurt the reputation of the component developer. When performing white-box testing for software components, we need to concentrate on two issues [5]: we must identify all possible external and internal scenarios that can occur for a software component, and incorporate them into testing. we must apply stricter adequacy criteria before delivering the component.

5.0 Systematic procedure for white-box testing of software components

The white-box testing is given less importance due to the factor that it requires high expenses in terms of time and resources. The amount of test code is greater than the development code, which raises the issue of the maintainability of the test code. Developers who usually writes the unit testing code, tends to overlook the faults and as well the criticism causes quarrel among the team members. To support the effective white-box testing, we need a systematic procedure for white-box testing with the aid of tools and process. The procedure should save the time and cost of the testing and easy the process of white-box testing. Lets investigate on white-box testing tools and methods to have effective white-box testing.

12

5.1

White-box testing tools

There are wide range of white-box testing tools [19] are available. The testing framework like JUnit, NUnit, provides developers more flexibility in developing the unit test cases and in the preparation of quality test suite. Few tools like JTest, TBurn, provides automation in test case generation and assisting the developer to manage the unit test cases. These tools also work as an extension of the testing framework. Tools like Clover, Cobertura, provide the information on the code coverage by the test suite, which acts as a useful information for the test case development. But, the tool does not evaluate the quality of the test suite and does not perform the mapping of the test cases with the source code at statement level. And the tools are not quite good in addressing the reuse of test cases for the regression testing.

5.2

Continuous Testing

Modern IDE's (integrated development environments) for the programming languages offers various features like code generation, continuous compilation, running the test suite, etc. But the most of the development time is wasted in rectifying the regression errors. A regression error that persists uncaught for a long time wastes more time to track down and fix than one that is caught quickly, for three reasons [20]: more code changes must be considered to find the changes that directly pertain to the error. the developer is more likely to have forgotten the context and reason for these changes, making the error harder to understand and correct. the developer may have spent more time in building new code on the faulty code, which must now also be changed. The idea of continuous testing [20], which uses real-time integration with the development environment to asynchronously run tests that are always applied to the current version of the code, combining the efficiency of asynchronous testing with the safety of synchronous testing. In this case, the developer never needs to explicitly run the test suite. The process can be tuned, prioritizing tests and parts of tests, which provide the illusion to the developer that the entire test suite runs instantaneously after every small code change, immediately notify the developer of regression errors. The continuous testing is achieved by constructing safe asynchronous model, where in the test execution is done on the current version of the code and the feedback is provided without the developer's explicit invocation during the development. This resembles to the TOTE (Test-Operate-Test-Exit) model of cognitive behavior [21]. In the TOTE model, plans consist of an Image of the desired result, a Test that indicates whether the result has been reached, and Operations intended to bring reality closer to the Image. Here, running the test suite is the Test, changing the code to make it work is the Operation, and a successful test run allows Exit. The test suite is an externally communicated Image of the developers desired behavior for the program. Thus, the idea of continuous testing uses the excess CPU cycle to the test programs while they are being edited and provides rapid feedback about test failures. Continuous testing

13

reduces the wasted time by 92-98%, a substantial improvement over the other approaches [20].

5.3

Regression model of white-box testing for software components

With the availability of sophisticated IDE and tools, assist in the rapid and effective white-box testing of the software components. But the environment, in which the component undergoes frequent changes, demands re-test to be done at unit level to ensure the component's quality. This poses the problem how to identify the components changes and analyze their impacts to its component test suite? What is its adequate re-test model and component unit test criteria? Developers and test engineers are lack of costeffective methods and tools to perform component change and impact analysis whenever a component is upgraded or modified. They also lack of cost-effective methods and efficient tools to support component re-integration, and test suite updates, and coverage analysis [22]. A re-test model will be helpful in the following ways: Present the dependency relationships between components at unit level Assist engineers to define re-test criteria and identification of reusable test cases Facilitate automatic test generation and adaptation to the modified component The test model, known as a CFAG (Component Function Access Graph) [23], is used to present possible component API-based function operation sequences. A CFAG model, representing the dependency and data flow among the functions within a component will assist engineers to perform API-based test sequences and coverage analysis. To investigate the change and impact analysis of the component, a dynamic view of component test model (known as D-CFAG) [23], is constructed. The component level changes like function and data changes can be captured, to refresh the test suite. But, this approach deals only with API level changes and not with the statement level changes. Also it does not focus on the coverage factor of the test cases.

5.4

Mapping of test cases to development code

Usually, test cases are created to test a module or function and the information of those mapping between test cases and module exists. But this information does not help in the test suite impact analysis, when a module undergoes modification. With the approaches discussed in the previous section, does not address the mapping of test cases to development code in a systematic way. In the component development, the modules present in a component might be associated with several test cases (test set) and coverage factor of a test set in the module varies, which implies that modifying few statements in the module, does affect only few test cases in the test set. In this situation, the developer either needs to execute those test cases as a regression suite or to modify only those test cases and its dependent (in case the logic of the module gets modified).

14

The figure below represents the scenario of mapping of test cases at module level and different coverage factor within the module.

Figure 3: Test cases mapping to development code. Depicts the mapping of test set to module level and test case at statement level

To address the above shortcomings, we need to develop the mapping of the individual test cases at the statement level of the development code i.e. a test case might involves testing a few statements in a module. This mapping information will be useful in the test suite impact analysis with respect to the development code changes and also helps in prioritizing the test cases. The mapping information assists the developer to have quality regression suite. In the process of continuous testing, it helps to run fewer test cases and saves the development time. Also it provides good code coverage information by the test suite.

6.0 Conclusions
In the software component development, white-box testing contributes the major effort towards the component quality assurance. Though the availability of the tools and IDE's make the process easier, developing a quality test suite is very important for an effective white-box testing. Tools should build a tight mapping of test cases to the development code at statement-level for an effective and continuous feedback in the testing environment. Tools should automate the most of the process like generating the test code templates, mapping of test cases, impact analysis, monitoring the test coverage details, test data generation, etc., since the white-box testing gets ignored due to lack of time and resources. So tools should play an important role in making the white-box testing effective. As the components are distributable, the information of the test cases need to bundled along with the component and the versioning of the test cases should be done accordingly (as mentioned in the Section 3.0). The developer should get enough information and metrics on the test suite, to assist in the process of test cases development and execution,

15

in order to develop a high quality software component with an effective white-box testing techniques. The future work of this study will on the development of a tool which will aid the developer to perform an efficient white-box testing process by collaborating with the exiting tools and utilizing the concepts discussed in this report.

7.0 Acknowledgements
I especially want to thank Prof. Deepak B. Phatak, as my seminar guide for his assistance, constant motivation, and encouragement to pursue my seminar work. I would also like to thank Mr. Avinash Awate for his valuable inputs on white-box testing of software components from an industry perspective.

8.0 References
[1] Boris Beizer, System Testing and Software Quality Assurance, International Thomson Computer Press, 1996 [2] Xia Cai, Michael R. Lyu, Kam-Fai Wong, Roy Ko, Component-Based Software Engineering: Technologies, Development Frameworks, and Quality Assurance Schemes [3] Booch, G., Software Components with Ada: Structures, Tools, and Subsystems, 3rd ed., Reading, MA: Addison-Wesley, 1993. [4] Heineman, G. T., and W. T. Councill, (eds.), Component-Based Software Engineering: Putting the Pieces Together, Reading, MA:Addison-Wesley, 2001. [5] Jerry Zeyu Gao, H.S. Jacob Taso, Ye Wu Testing And Quality Assurance For Component Based Software,Artech House Boston, 2003. [6] Brown, Alan W. & Wallnau, Kurt C. Engineering of Component-Based Systems, 715. Component-Based Software Engineering: Selected Papers from the Software Engineering Institute. Los Alamitos, CA: IEEE Computer Society Press, 1996. [7] Gao, J., Challenges and Problems in Testing Software Components, Proc. of ICSE2000's 3rd International Workshop on Component-Based Software Engineering: Reflects and Practice, Limerick, Ireland, June 2000. [8] Gao, J., et al., On Building Testable Software Components, Proc. of 1st International Conference on Cost-Based Software System, 2002, pp. 108-121. [9] Stocks, P. A., and Carrington, D. A, A Framework for Specification-Based Testing, IEEE Transaction on Software Engineering, Vol. 22 (11), IEEE Computer Society Press (1996), 777-794. [10] Wang, Yingxu, King, Graham, and Wickburg, Hakan, A Method for Built-in Tests in Component-based Software Maintenance, Proceedings of the 3rd European Conference on Software Maintenance and Reengineering, (1998). [11] ANSI/IEEE std 610.12, IEEE Standard Glossary of Software Engineering Terminology, 1990.

16

[12] Grady, R. B., Successful Software Process Improvement, Englewood Cliffs, NJ: Prentice Hall, 1997. [13] http://www.scism.sbu.ac.uk/law/Section5/chap3/s5c3p23.html - Retrieved from www on 11-Feb-2007 [14] http://www.onjava.com/pub/a/onjava/2003/05/07/blackboxwebtest.html - Retrieved from www on 11-Feb-2007 [15] McCabe, T., A Complexity Measure, IEEE Transactions on Software Engineering, December 1976. [16] P. G. Frankl, E. J. Weyuker, An applicable family of data flow testing criteria, IEEE Transactions on Software Engineering, Volume 14, Issue 10, October 1988 [17]http://www.testingeducation.org/conference/wtst4/ChangLiu2005-WTST4-dataflow-testing-full-paper.pdf - Retrieved from www on 11-Feb-07 [18] Hong Zhu, Patrick A. V. Hall, and John H. R. May, Software unit test coverage and adequacy, ACM Computing Surveys (CSUR), Volume 29, Issue 4, December 1997. [19] http://www.testingfaqs.org/t-unit.html - Unit Testing Tools [20] David Saff, Michael D. Ernst, Reducing wasted development time via continuous testing. In 14th International Symposium on Software Reliability Engineering, pages 281 - 292, 2003 [21] G. A. Miller, E. Galanter, and K. H. Pribram, Plans and the Structure of Behavior, Holt, Rinehart and Winston, Inc., 1960. [22] J. Gao, D. Gopinathan, Quan Mai, Jingsha He, A Systematic Regression Testing Method and Tool For Software Components. In Proceedings of 30th Annual International on Computer Software and Applications Conference, Volume 1, pages 455 466. IEEE Computer Society, September 2006. [23] Jerry Gao, Raquel Espinoza, Jingsha He, "Testing Coverage Analysis for Software Component Validation". In Proceedings of the 29th Annual International Computer Software and Applications Conference (COMPSAC05), 2005

17