TEST AUTOMATION FRAMEWORK & DESIGN For Xxxxx project

Author: xxxxxx

Abstract

Automation Testing is defined as developing and executing tests that can run unattended, comparing the actual to expected results and logging status. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process.

Test Automation is the use of strategies, tools and artifacts that augment or reduce the need of manual or human involvement or interaction in unskilled, repetitive or redundant tasks.

Test Automation scripting is akin to development and programming; and hence some practices used in software development apply. This includes a proper Automation Framework, code standards and reviews. A good Automation Framework is indispensable for an organized test automation activity. A test automation framework is a set of assumptions, concepts, and practices that provide support for automated software testing. The correct automation framework is an essential factor determining the success of the test automation activity. The framework refers to the overall environment we construct to execute our tests. This paper attempts to present a general idea about the automation framework which will be followed in xxxxxx.

Page 2 of 14

TABLE OF CONTENTS

1. Introduction................................................................................................................................................................4 2. Test Automation Framework.....................................................................................................................................4 3. Automation Test Scripts .......................................................................................................................................7 4. Data Repository......................................................................................................................................................7 5. Function Library....................................................................................................................................................8 6. Script Management................................................................................................................................................9 7. Test Results..........................................................................................................................................................10 8. Output Files..........................................................................................................................................................10

Page 3 of 14

1. Introduction
As the Internet AGM is developed using Web and J2EE Technologies, and since there is no interaction between client and server, Java add-in is not required. Automation is going to be done using the default add-ins of QTP which is Web, ActiveX and VB. We are going to follow Hybrid Framework to ensure that the automation is being done in a best way. We are going to pull some of the essential and useful features of each automation framework and implement in one framework which is Hybrid Framework. We are going to follow Descriptive Programming to ensure that QTP replays fine on all the environments and the object identification is always successful Wherever object learning is essential, objects will be learned to make sure that the automation is completely satisfies the Business Requirement. Automation will be done based upon the following information: Business Requirements. Use Cases.

1. 2.

3. Test Cases.

2. Test Automation Framework

Page 4 of 14

A Test Automation Framework is a set of assumptions, concepts and practices that provide support for automated software testing. It stipulates various guidelines regarding script organization, and coding standards. It should give the automation engineers an idea as to how to code and organize their scripts. This framework will ensure the below advantages:

Reusability of scripts. Faster development of new scripts. Ease of adaptation to changes in the Application. Less human involvement. Complete parameterization of Test Data. Complete handling of exceptions.

The main concept behind this framework is to reduce all test cases to their most fundamental tasks, and write User-Defined Functions, Business Function Scripts, and "Subroutine" or "Utility" Scripts which perform these tasks independently of one another.

Page 5 of 14

FUNCTION LIBRARY AUT Common Function Library Wrapper Functions MANUAL TEST CASES

OUTPUT FILES

DATA REPOSITORY Input Data for Automation Script AUTOMATION TEST SCRIPTS

RESULTS Individual Test Run Results Master Script Run Results

Environment Data

Temporary data files

SCRIPT MANAGEMENT Master Driven Script

Exception Handling & Recovery

Fig.1 AGM - Test Automation Framework

Page 6 of 14

The figure shows a number of blocks, each an integral part in the overall framework. Each block and its role in the framework are explained below. A few blocks appear in dotted lines, as for instance, the block Output Files. Any block appearing in dotted lines indicates that it is optional, and gains a meaning only if there is a need in the project. They have also been included here, since automation engineers involved in relevant development of scripts will identify the use of it. The above framework, in fact, shows the folder structure that can be implemented in the test automation project i.e. each block and its sub-blocks can also be considered as a folder and its sub-folders where the files with corresponding information is stored.

3.

Automation Test Scripts

This is shown as the central module in the figure. These are scripts developed using

QTP. Descriptive Programming will be performed for the entire automation. This script is usually enhanced with additional statements to make it more generic, remove all hard-coding of values and add may be, additional functionalities, as per the requirement.

4.

Data Repository
Most scripts will be data-driven i.e.; the scripts will have the capability of using

variable data. In addition, in most cases, the scripts should also be capable of working in different environments. This particular block deals with all these aspects of the scripts. This encompasses 3 sub blocks Input Data for Test Cases, Environment Data and Temporary Data Files. The format of the data files is usually excel sheets; it can even be text file. Input data for Automation Script is basically the data required for execution of each test script. Since scripts will be data-driven rather than hard-coded, all required data for various input fields etc. will be fetched from these data files. Depending on the complexity of the AUT (Application under Test), the automation testers may have to decide whether to maintain all test script input data in a single file or have a separate input file for each test script.
Page 7 of 14

The contents of Environment Data may depend on the AUT and the test scripts. In some cases, this may be the URLs of applications, or it could be FTP or Telnet IPs. In other cases, it could be the information pertaining to the screen resolution, or the date formats to be used, or database connectivity information i.e. any information that pertains to the setting up of the AUT/Script execution environment. It would be ideal to have just one file for storing all the environment data. However, again the actual number of files that need to be maintained will depend and will be decided. The next block shown in the figure is the Temporary Data files. This block is optional. In some scenarios, it may be required that you are saving a huge database query result or FTPing some data file from the server which in turn will be used by the remaining code. This is not the end result, nor will this data be required after the processing. It can possibly be safely deleted after the processing. These are the temporary data files, generated in the process of script execution. If the test needs to create a temporary file, these temporary files can be provisionally saved in some common folder.

5.

Function Library
This block will hold a set of common functions that will be called by the script. This

will be a set of reusable functions that will be called by the test case scripts, as and when required. Under this block, shown 2 sub-blocks AUT (Application Under Test) Common Function Library and Wrapper Functions. AUT Common Function Library will include all the reusable function that is specific to the AUT. For instance, login common function, logout common function etc. Wrapper Functions are also reusable components but refer to the tool in hand. This is optional, and most often not developed in many an automation project. A wrapper function

Page 8 of 14

is any function written to implement the functionalities provided by the test tool. Initially wrapper functions will call the function from the tool directly. Over time, the wrapper functions may be modified to add functionality not natively present in the test tool, or to write a workaround if the function native to the tool ceases to work properly.
6.

Script Management
This block will hold all other auxiliary scripts that will be needed for executing the

test scripts. In other words, this blocks components will help manage the test scripts. The figure shows 2 blocks under this Master Driven Script and Exception handling & Recovery. Master Driven Script refers to the startup script which will call other scripts to test the entire functionality of the application. Exception Handling & Recovery is the next block in the Script Management. This block deals with the various errors that might crop up while the script execution proceeds and methods to handle these. QTP has its own built-in exception handling / recovery management routines. The automation tester will only need to customize these functions in accordance to the error scenarios associated with the AUT. One of the primary objectives of automation is to run the scripts unmanned. This can happen only if proper exception handling is in place. This will ensure that even if an error message does pop-up in the midst of execution, this will be automatically handled by the test script and the script will either continue execution or gracefully exit rather than abruptly stopping the execution (this will happen if there are no exceptionhandling routines associated with the test scripts). Most often exception handling will involve invocation of a clean-up script which close any open files, any open database connections etc.

Page 9 of 14

One important aspect in this respect is to know in advance what all errors / exceptions might occur and how to handle these. This can be known only if the automation team has a good knowledge of the AUT as well as the automation test case suite. The automation tester must have manually executed the automation test case suite, for him/her to have gained this knowledge.
7. Test Results

QTP will generate result files which will have the log of the script execution. In some cases, this may not suffice and the tester may write code for the generation of customized log files. Two types of log files can be expected Test Case Log file and Test Run Log File Individual Test Run Results file logs all the steps and results of each test script i.e.; number of steps that passed and failed in that particular script. Master Driven Test Run Result file logs the result of the entire run. A run can include execution of all test scripts, or a few selected test scripts. The result shows the status of the run i.e.; number of passed and failed test scripts.

8.

Output Files
This entire block has been shown as optional. Usually the expected output of a test

automation will be the test results which gives a detailed account of what all passed and what all failed the obvious result of a testing activity. However, in some cases, there might be a few output files that get generated as a part of processing in the AUT. These may perhaps be

Page 10 of 14

used only for some verification. So, in case these files are required, they could be placed in a separate output folder. For instance, if the AUT is a billing application, a customers bill will be an obvious output of the AUT. There may be a test script which needs to check the format of this bill. If the bill is not needed after test script execution, it can be placed under temporary data files. But in case it is required, say as an input to some other module or for manual verification, it can be retained as an output file in say, an Outputs folder.

Page 11 of 14

9. Automation Design
Master Script Test Case ID: xxxx Step 1: Call component Ref. <<Component Script>> Report to result Ref. << Reporting results>> Step 2: Test Case ID: xxxxx Step 1: Call component Ref. <<Component Script>> Report to result Ref. << Reporting results>> Step 2: . . . Component Script
Input Parameter

{Coding .. }

Output Parameter

Library Functions

Recovery Scenarios

Input File Data Consists of all input data which should be utilized as parameters

Reporting results Test IDStepExpected ResultActual ResultResult11----------------Pass12-----------------Fail 21------------------Pass

Fig.2 AGM - Test Automation Design

Page 12 of 14

Input Parameter Whether to verify fields or to enter the data. Fields to be verified or data to be entered.

Output Parameter True Component passed. False It needs to send flag number and error description.

Master Script: These are identified based on Use Cases, so we have 24 master scripts. Each master script will create a result sheet and the result sheet will contain the data which is given in the diagram. It will also return the number of test cases passed and the number of test cases failed with Test Case ID in a separate sheet and the same will be modified by other master scripts. Component Script: These are identified based on reusability. Component script will get parameters from the Master Script and will execute. If the execution is success, it will send a output parameter True to the Master Script. If the execution is not success, it will send three output parameters 1. False, flag, and error description based on that Master Script will print on the result sheet. Components List:

C mo e t L t o p n n is

Data Table:

Data Table

Reusable Functions:

Reusable Functions

Recovery Scenarios:

Page 13 of 14

R c v ry S e a s e o e c n rio

10. CONCLUSION
Thus automation framework outlines how to organize scripts and what is required in addition to your basic script for a successful automation. The framework gives an indication of how the entire activity can be modularized and how the various components interact with each other. The above explained framework is just one of the many possible, and perhaps suitable, frameworks. It is equally necessary for the entire team to understand the adopted framework and the significance of each block and relationship between the blocks so that all will develop code which adheres to the framework.

Page 14 of 14