Sie sind auf Seite 1von 6

Automation & Performance test progress report

DALE Chris - 9 June 2011

This report details the achievements, progress and planned work for the Technical Test area. Details of this report and past reports are collated here: http://itzone/Servstratdesign/SS/change/PerfTesting/SitePages/Home.aspx

Period report covers:

09 April 2011 09 May 2011

Issued to: Eric Chapman, John Bennett, John Goodyer, Paul Smith, Stuart O'Loughnane, Kath Watson, Mike Evans, Dominic Hyland, Malcolm Peck, Jamie Dalton

Achievements & Progress

Visual Studio Set Up: Testing / Developer collaboration. o TFS Source control created for the Hewitts project allowing both testers and developers to access the performance tests. o Test and Development have been working more closely together on the Hewitts project for performance testing and have identified some issues that needed to be fixed on the site. Consensus is that this has resulted in defects and issues being resolved earlier. Have requested that user group G-Performance-Test-U-C (Chris Owner) be added as a Performance Monitor User on all of our TEST & PREPROD environments. This group contains tech test users, test accounts, VS2010 controller user etc. Meeting with Roger Faithful booked to move this forward. Azizah has now presented to the test team how MTM has been used to plan, test and track defects on the Aon Hewitts project, this was a really good overview. Colin Beales (Microsoft) has delivered a presentation covering tests, linking to requirements, defect raising and tracking, reporting. Working with Jamie Dalton I have been able to make significant progress on the set up of Visual Studio 2010 for the purpose of Performance Testing. We have encountered many challenges along the way and extremely happy that we have been able to overcome these so quickly and efficiently. We now have a basic test rig, made up of: o Controlling server o Five client machines. o This rig now needs to be ramped up to ensure we are able to support a whole range of projects. o A database is required to collect the performance data for projects and to store baseline results; this database is being built by OPS (with our guidance) this week. Stuart, Mike & I have been discussing a collaborative approach to running a C# course and have now been able to secure an in house course provided by QA for 10 delegates. This has proved to be extremely good value for money, potentially bringing down the cost per delegate from 2500 to 500.


Page 1

AON Hewitt: (VS2010 proof of concept) As mentioned in last months report, it has now been confirmed that the AON Hewitt PREPROD environment needed the Ram increased from 4 GB to 8 GB. This has now been resolved, although I have not been able to complete another full test due to the availability of the database mentioned above. Initial results look to have improved things.

Automated Testing No movement in this area from the last report the following two points need to be picked up when time permits. The reusability of the Embassy data migration and how this work could be reused to populate a database for Test Automation & Performance requires an environment to be built to enable this initiative to be taken forward. FNZ Regression tests have been put on hold due to the environment reliability issues and recruitment of the senior Test Manager.

AWWIPS I have fed into Erics DSDM Test Approach to suggest the tools we will be using to aid and support his system test team. Automation will be required to build tests during time boxes to ensure that time boxes are not filled up by manual testing time. We will be using VS2010 to achieve this. These tests should be delivered as part of the project (time box) output. John Bennett will be leading this activity. Performance Test will be required. I have met with the business stakeholders briefly to understand the non-functional requirements. Intend to engage with them further to agree the business scenarios and agree approach to various activities under Performance Testing. The VS2010 toolset will be used for this, led by Chris. Tool evaluation for HTML Validators, Browser checker and Link checker software is underway. I have approached contacts in other AXA SBUs for input on what they use and have had a very good response from this, in particular from Kevin Atkinson at Swiftcover. Scott Readings and I will be discussing this to ensure there is no Development / Testing overlap.

Belfast As mentioned in last months report no Automated Testing will be required. Performance test is a requirement for this project. NFRs have been queried and need enhancing. Browser checker and Link checker software should be used to help identify problems early in the development / test process. (Should have lessons learnt from AWIPS by then).


Page 2

Other Business: Chris has had 1 week paternity leave this month. This is reflected in the progress made.

Planned Work
Visual Studio Set Up / AON Hewitt Run the performance and load scenarios on AON Hewitt following confirmation that the Test Rig database has been built by OPS. Analyse the results from the AON tests and produce a final report. Chris to cross train John Bennett on the Performance activity done with Visual Studio 2010. John Bennett has been self-teaching C# and has been applying this to Visual Studio. This will help with the deliverables required for the AWIPs project. Begin the long process of establishing a framework for codedUI testing in Visual Studio 2010, identify where to store things such as function libraries, stored procedures and functions, how to report results, document procedures and processes (although light touch, but enough to cover business continuity). John to cross train Chris in findings so far in Visual Studio CodedUI. John and Chris to work together during the AWIPS project to produce CodedUI, Performance Tests and support for the other tool sets being investigated for this release.


Page 3

Time Booking
In the short term, time will be required to set up testing frameworks for the use of C# in Visual Studio 2010, where to store things, how to report results, reusable procedures, document procedures and processes (although light touch, but enough to cover business continuity). As mentioned in my Automation presentation this is development process to create a computer program that tests a software delivery and as such takes time to achieve. We should be prepared for the fact that the next two projects will not receive great gains. We intend to meet with John Goodyer to discuss the best way forward to achieve this. To cover the month of June and July (8 weeks) commencing 6th June we propose the following. Please let me know if there are any objections to this, if so please suggest an alternative.

Tester John Bennett John Bennett John Bennett John Bennett John Bennett John Bennett Chris Dale Chris Dale Chris Dale Chris Dale Chris Dale

Project / Activity AWWIPS AWWIPS September BAU C# Course / BAU Holiday AWWIPS AWWIPS BAU Belfast C# Course / BAU

Number of Days 7 14 7 3 4 5 7 14 7 8 4

Category D D R O D D O D -

Benefit CodedUI tests delivered per time box Access test tools and confirm purchases Run automated regression pack Set up and configuration of CodedUI tests in VS2010 for all future projects Progression toward the goal of one tool set for the department CodedUI tests delivered per time box Access test tools and confirm purchases Continued development of technical test strategy and implementation of VS2010 Consult on NFRs for project and agree testing activities for later in the project Progression toward the goal of one tool set for the department

D = Direct Use R = Regression Use O = Generic work and development of technical test for the future


Page 4

Issues & Risks

Date Risk Description Mitigating Action Status Owner Severity Date Closed

te Raised Ref ised


08/04/2011 To enable competent codedUI tests using VS2010 a C# Course will be required to give the testing area the ability to code these tests.

An introduction to C# courses has been requested, inc costs.

Closed Course booked for 12 July Open

Chris Dale / Eric Chapman




08/04/2011 Until the positions in the OD are filled, resource will have to be allocated on a priority basis. 08/04/2011 The issue of what the tech test team should book their time against should be resolved.

Access projects and other needs in line with resource availability. Time should be allocated across projects.

Chris Dale / Eric Chapman





John Goodyer / Stuart




Page 5