Sie sind auf Seite 1von 11

1

Success through Test Strategy @ NIIT

Success through Test Strategy @ NIIT


Neeti Churamani Kumud Iyer
Every software project is different in nature and a lot of its success depends on the type and kind of verification validation. We have tried to capture some of the challenges faced by the team, which was doing a field remediation project. This project was significantly different from Y2K projects handled, as in the latter; generally the existing field was expanded in-place. The difference in this project was: The impacted fields were added as new entities in existing Datasets. Moreover, Y2K projects were only for date field expansion, whereas this project was on five different business entities and related business logic. The data carried forward in the newly created fields was consistent with the legacy values. This was handled by changing the processing in the impacted routines and by creating Bridge routines for first time input datasets. A special treatment logic was created for the cases where legacy field value limits were exceeded. We have tried to bring out the experiences, learning and in process improvement in this project lifecycle. This paper focuses on sharing a success story of field remediation project, through effective Test Strategy and benefiting the community with NIITs experience.

Abstract
The client, a leading Media Research Company, is a major provider of television ratings and audience estimates information services, serving national and local customers including television networks and affiliates, independent stations, syndicates, cable networks, cable systems, advertisers and their agencies. The broadcast television industry is in the process of shifting its method of signal transmission from analog to digital. As a result of this decision, the client evaluated their information processing systems and found the need to initiate a Field Expansion project to accommodate advances in the video broadcast industry surrounding digital transmission standards.

As digital stations begin broadcasting, they work to stay ahead of the industry by assuring the ability to measure and appropriately credit viewing. As part of the analysis done for Digital Conversion, the number of applications that were found to be digitally impacted was thirtysix and the project Conversion for Digital Signal Analysis was initiated. NIIT was awarded this project in June 2002, as it showed the following strengths Its experience in the technology viz. PL/1, SAS, Easytrieve, Assembler on IBM mainframes Experience in Y2K projects Established Testing and LMM practices

Background
The client provided a Conversion Strategy that required five business entities to be digitally impacted along with business logic. This was a critical project as it was first of its kind for NIIT (as this was different from the Y2K projects done in the past) and opened a new domain of Media Research for our organization. The applications to be remediated were in various mainframe languages like PL/1, Assembler, Easytrieve, SAS and TSO/ISPF etc. The challenge was to test the application without having enough time to gain indepth functional knowledge of the Critical business applications. It was a highly complex system processing huge amounts of data daily, in order of about 300 million data items a day and thus had very stringent batch window requirements for the converted system. Out of the various levels of the testing, Regression Testing was the only reliable way to ensure that the remediation process did not inadvertently introduce errors into the existing functionality within the Business Rule as we had upgraded these rules as per the Conversion Strategy. The objective of the Testing Phase was to ensure that the software components of various impacted and non-impacted modules of the applications worked as intended after the Code Remediation

The Challenge
An automated tool was used which scanned the JCL and PL/1 programs and generated cross-references of programs to logical files and thence to physical files. The scope of integration testing was to test the relevant jobs, of impacted applications, to ensure that all interfaces and bridges functioned correctly even after introduction of digitally converted routines within various jobs. The workflow of the Field Expansion project was as follows: The project team was divided into two groups Development team and Testing team. The Development team was responsible for: Impact Analysis of Digitally impacted routines Creation of Unit and Regression Test Plans Code Remediation Review of Remediated Codes 1st round of Unit and Regression Testing Integration Testing: 1. 2. 3. 4. 5. phase and hence the functionality of the integrated application was retained. This phase was recursive and was to be repeated, as many times as there were levels of testing. Since this project was not like typical application

development/enhancement, hence the types of testing which were decided for this were: This was done for every routine by verifying that the digital fields, which got added as a part of remediation, contained correct values. For example, for a field called station code which in legacy had a value of 9998, after remediation the corresponding digital station code value was 0000009998. In case of a station code exceeding a value of 10,000 the legacy values was set to 0. The scope of regression testing was to compare the output of integrated units (at various levels) with the base lined production runs. Unit Testing: Regression Testing:

Testing Strategy
3

Testing Strategy
Development Team Preparation of UT/RT Test Plans Impact Analysis Code Remediation And Review Defect Closure Delivery of Routines and Test Reports 1st Round Unit Testing + Regression Testing Review Test Plans Prepare Integration Plans Defect Reporting 2nd Round Unit and Regression Testing Final Inspection By Quality Reviewer Testing Team Defect Verification Base Line Production Run Integration Testing

Figure 1 2
Success through Test Strategy @ NIIT Success through Test Strategy @ NIIT

3
The Testing Team was responsible for: Baseline production run Review of the Unit and Regression Test Plans created by development team 2nd round of Unit and Regression Testing Creation of integration Test Plans Integration Testing The client did the Acceptance testing with support from the testing team. The various steps followed during the course of the Testing process are as follows: The developers carried out the 1st round of Unit Testing using copies of production data to ensure that the routines gave the desired output after remediation and using expanded datasets containing expanded field values. Boundary conditions of expanded fields were tested to ensure compliance to Conversion

Strategy. 2nd Round testing was done by the Testing team to ensure correctness. Identify input / I-O / output files for every digitally impacted routines. The automated tool that analyzed JCL and PL/1 source codes was used in the same. Baseline Production jobs containing impacted routines. In this step, copies of Production datasets were taken and baseline runs were done of jobs and all the datasets of these jobs were preserved. These were later used for comparisons with Digital run datasets. Digital run In this step, copies of baseline run input datasets was created and the digitally converted routine was executed. The output files created were preserved with different names indicating that these were digital run outputs. Comparison of Baseline run and Digital Run In this step we carried out an intelligent comparison of the baseline files and the regression files. The Regression Testing was considered to be complete only all the output datasets of the two runs compared successfully. Check that all the digitally impacted routines were successfully Unit and Regression tested. Setup up copy of the production jobs by setting up input datasets of nonimpacted as well as expanded datasets for impacted routines. Execute the new production job containing the digitally 1. 2. 3. 4. 5. Unit Testing Process: 1. 2. Regression Testing Process: 1. 2. 3. 4. Integration Testing Process: 1. 2. 3. impacted and nonimpacted routines. This testing was conducted as part of the Baseline run and Regression Testing cycles where the CPU usage time of the production baseline was compared with that of the digital routine. In case of routines where CPU usage was found to be exceeding production runs substantially, routines were optimized to ensure that the jobs run time was retained closely equivalent to the original jobs. One of the success criterias for the regression testing was that the time increase for an individual program should not exceed 10% and the time increase for the overall job should not exceed 2%. All the above-mentioned cycles of Testing were repeated till all identified problems and issues were tracked to closure. Performance Testing: One of the major process revisions that were made to the Testing Phase was the merging of the Unit and Regression Testing cycles. In Phase I, we first completed all Unit Testing, then Regression tested all those impacted routines and then Integration Testing was completed at a job level. This posed a few concerns: In case of an error detected at the Regression Test level, the developer had to re-run all the Unit level test cases. The Testing Team had to re-execute the second round of Unit

Testing. As the routines were clubbed during Regression Testing, it was cumbersome to detect the exact location of the error. To resolve this, during the course of a Causal Analysis and Resolution, the project team decided that each impacted unit be regression tested independently at the unit level. This helped in cycle time reduction by catching the record discrepancies at unit level itself in the subsequent Phases (Productivity Measure Annexure A). Over and above this one round of regression testing was also done at logical integration of units.

Testing Process Improvement


Use of Independent Testing Team The testing team was responsible for the review of the Unit and Regression Test Plans, creation of Integration Test plans and to ensure vigorous and impartial testing of the remediated routines and applications. Since this team was dedicated to the project for complete duration, they were fully aware of the Conversion Strategy, business domain and system requirements of the Digital

Critical Success Factors


4

Usage of Tools
To assist the Testing team meet the challenge of ensuring Digital Expansion without affecting the applications functionality, the following tools were used: CompareX: This is an IBM mainframe tool that was used for comparing the output files of the Legacy and Digital runs. Strobe: This tool is helpful during Performance testing. This was used for identifying if any of the impacted units had been inadvertently affected in terms of performance after its remediation. Load Data: This tool (a simple PL/1 program) was used to convert original datasets into digital datasets during testing. The need of this tool was felt due to: Voluminous datasets Datasets with variable record lengths. These could not be digitalized using File Aid. Data Dictionary generator In-house developed tools that analyzed the JCL and PL/1 source In-house Tools: Conversion project. The project team worked by linking to the customers computing environment and utilized the tools available. This approach helped ensure that the test environment of the applications was similar to Production. The Knowledge Portal is a comprehensive information and knowledge management package. It enables organizations maximize their efficiency by providing generative information and allied service to create, share and apply knowledge within the organization. The use of the KP helped the offshore team communicate easily with the Onsite team as well as directly with the Client. Regular Weekly Conference calls and Status Reports with the client help address issues/concerns in the early stages of the project life cycle. For the testing cycles carried out by the project team, the customer allowed access to production data by creating copies of those datasets. This helped the testing of the impacted applications by ensuring that all possible permutations and combinations of data were tested right from the beginning of the Testing cycle. Potential performance issues were thus detected early on at the testing stage itself rather than in later phases.

Leveraging the Clients Environment Collaborative Tools Access to Production data for Testing

Success Measures
1. 2. 3. The project was delivered On Time. Absolutely smooth transition to production; the applications delivered in Phase I & II are live in production while the last few applications of Phase III are slated to go live in January 2004. (Defect Measures Annexure A) The performance of the remediated applications in terms of CPU Time was as per the criteria specified hence ensuring that the customer didnt have to make any changes in the original job schedules. codes and created a data dictionary of programs, logical files, physical files that was useful for detecting impact, testing. Note: The dip in the Productivity figures (in hrs/KLOC) indicates an improved effort.

Annexure A Annexure B Figure 3


Figure 2 Success through Test Strategy @ NIIT For more information, visit at www.niit-tech.com or contact it.solutions@niit-tech.com

Build your own FREE website at Tripod.com Share: Facebook | Twitter | Digg | reddit

NIITs Strengths
Opportunities to Build a Future in the e-Commerce Space
Get eReady or face e-Xtinction. Thats the belief at NIIT. Today, were eNIIT, a company thats moving radically into e-commerce and changing itself internally to deal with whats to come. NIITs strengths in e-commerce:

NIIT has a vast repertoire of products and services, especially centered around Internet and e-commerce solutions NIIT has a broad range of customers it is e-transforming so they can face the challenges posed by the 21st century NIIT has an edge in e-learning/distance learning (a major part of e-commerce activity) through its learning solutions NIITs portfolio of e-commerce solutions includes remote services/e-services

With NIIT you can build a career at the forefront of the e-commerce revolution.
If youre someone with an eye on the future - someone who wants to be a part of the brave new world, select NIIT. The choice is clear. As a player in the e-commerce domain, weve built technologies which have re-usable components. With these, we can create what

are called 90-day solutions for companies. Therefore, whether a company wants to set up a portal to do business on the Web, or establish a full fledged shopping complex in cyberspace, NIIT is geared up to do the job.

- NIITian, Vijay Thadani President & CEO

Opportunities to work with the world's best, most reputed organizations


Strengths NIIT has in the customer space:

A customer list that includes American Express, Bank of America, British Airways, Citibank, Deutsche Bank, Hitachi, Honda, HP, IBM, Microsoft, NEC, Sony and Toshiba among others A vast community of satisfied clients that are leaders in their respective industry segments A strong global customer base

A career at NIIT will offer you a chance to:


Experience a variety of different environments across businesses, industries and countries Work with new economy businesses that will be at the forefront of the ecommerce revolution tomorrow Work with some of the most hot companies, both at home and abroad Be a part of our training, software development, systems integration or ecommerce initiatives Work either on-site or off-shore at prestigious customer sites

If youre someone who wants to build a career based on exposure to global class environments, select NIIT. The choice is clear. Joining NIIT gives you the maximum opportunity to start on a rewarding career where you also have the opportunity to work with some of the best global companies.

- NIITian, Arvind Thakur President, Software Solutions

Opportunities to Enjoy a Unique Work Culture

NIITs strengths at work:

NIITs culture is open and informal and conducive to innovation NIITs key strength are its 4,000 plus people who form the knowledge community NIIT believes in sharing knowledge, which means access to both people and resources that can help expand your knowledge base NIIT has a unique culture which stems from the top and percolates all the way down to the grassroots NIITs brains trust comprises such industry gurus as Rajendra S Pawar, Vijay Thadani, P Rajendran, Arvind Thakur and Sugata Mitra. The companys innovation, drive and vision flow from these power houses

Therefore, if youre someone who believes work culture determines personal growth, select NIIT. The choice is clear.

Copyright 2000 NIIT Ltd. All rights reserved

Careers | Investor Relations | Sub

ut NIIT Awards and Ranking

Awards and Recognition 2011

RELATED CON
NIIT Factsheet All you wanted to know

about NIIT at a glance


ng

opment

aining

Earned fifth consecutive appearance in the Top 20 Companies in the Training Outsourcing Industry ranking Earned fourth consecutive appearance at #63 in the Training Top 125 ranking from Training Magazine, the leading business publication for learning and development professionals. Listed #31 among the LearningElite in Chief Learning Officer magazines prestigious new ranking. Awarded the Best Event Marketing Campaign under $100K, 2011 by the American Marketing Association (AMA) Atlanta Chapter Ranked #8 overall in the HRO Today Baker's Dozen Customer Satisfaction Ratings for Top Learning Providers, 2011

2010

Earned the Silver for Best Results of a Learning Program for the Brandon Hall Excellence in E-learning Awards 2010. Earned the third consecutive Gold for Excellence in Content at for KFC Virtual Worlds at the Chief Learning Officer Learning in Practice Awards. Named to the Top 20 list of Training Outsourcing Companies, according to Training Industry, Inc. Earned # 1 position in Quality of Service and #2 Overall in HRO Today magazines Baker's Dozen Customer Satisfaction Ratings for Top Learning Providers 2010. Received an honorable mention for learning and talent management solutions at the Bersin Impact 2010 Conference. Given a Citation for the HP Global Sales Accreditation Program jointly with HP, HD Productions and Systems Documentation, Inc. at the ASTD Excellence in Workplace Learning and Performance Awards. Earned the Sales Impact award at the Markie Excellence in Marketing Awards, 2010. Earned third consecutive appearance at #67 in the Training Top 125 ranking from Training Magazine, the leading business publication for learning and development professionals. Named to the Top 20 list of IT Training Companies, according to Training Industry, Inc.

2009

Earned second consecutive Gold for Excellence in E-learning for Diageo IS Training at the Chief Learning Officer Learning in Practice Awards. Earned the Bronze for Best Innovation in Learning Technology for the Brandon Hall Excellence in E-learning Awards 2010. Earned second consecutive appearance at #85 in the Training Top 125 ranking from Training Magazine, the leading business publication for learning and development professionals. Named to the Top 20 list of Training Outsourcing Companies, according to Training Industry, Inc. NIIT USA awarded Tech America Spirit of Endeavor Award for Leadership in

Technology Education, 2009. Named to the Top 20 list of IT Training Companies, according to Training Industry, Inc. Ranked in the Top 10 of The Black Book of Outsourcings annual ranking of Top Training and Learning Business Process Outsourcing Vendors, 2009. Awarded the first ever Marketing All-Star Award for excellence across all categories at the Markie Excellence in Marketing Awards in 2009.

2008

NIITs Learning Administration Services were given a Vendor Innovation Award by top learning and talent development analyst firm, Bersin & Associates. Awarded the Business Superbrand 2008 status for the second time by the Superbrands Council, US (the first one was for the period 2003-2005). Ranked as Top Training Company 2008, by Dataquest magazine, Indias leading ICT publication group Cybermedia. NIIT, USA awarded the Gold for Excellence in E-learning Award for Allstate Auto Quoting by Chief Learning Officer magazine as part of its Learning in Practice Awards, one of the industrys highest honors. Ranked among the Top 20 Companies in the IT Training Industry by TrainingOutsourcing.com. Recognized by UNESCO for innovation in ICT in Education. The company (HiWEL) earned the coveted Digital Opportunity Award by World Information Technology Services Alliance (WITSA). Ranked among Indias Most Trusted Services Brand in a survey by India's one of the leading financial daily- The Economic Times Brand Equity Survey. Only Indian Education company from India to be featured in "The BCG 50 Local Dynamos." Ranked among the Top 125 companies in the US with the most outstanding training organizations, according to Training Magazine.

2007

Earned the Society of Insurance Educators Innovation Award for Allstate AutoQuoting, 2007. Ranked amongst Indias Most Respected Companies in the IT sector in a survey done by Business World magazine. Ranked as Indias Most Customer Responsive Educational Institution per Avaya GlobalConnect Customer Responsiveness Awards for the third consecutive year. Awarded the Indian Franchisor of the Year Award by Franchising Association of India. Rated among Training Magazines Top 125 elite companies worldwide, for its innovative training practices. Ranked among the Top 10 Business Brands since 60 years of Independence, by leading Indian magazine The Week.

For a complete list of awards and recognition earned by NIIT, download the complete list here.

Join our Ne

sion

About Managed Training Services Elements of MTS Strategic Assessments Engagement Management Governance Improvement Global Capabilities Why Our MTS For Associations For Higher Education For Publishers For Enterprises For Technology Companies

Solutions for Your Business & Industry Learning Administration Learning Content Learning Delivery Learning Management Systems Learning Technologies Advisory Services Catalog Content Demand Generation Student Retention Virtual Labs

Resource Center Blogs Podcasts Videos Brochures Reprints Webinar Case Studies White Papers

News & Events News Newsletter Media Kit Events WorldWide News

Co

Lo

Co

ion

Su

Su

Fe

pment

Legal Notices | Terms & Conditions |Privacy Policy | Site Map 2011 NIIT | 1050 Crown Point Parkway, 5th Floor Atlanta, GA 30338 |E: businessimpact@niit.com | P: 770-290-6000 | F: 770-551-929