2015.03.11 Testing Trends Slide Deck EMEA

Testing Trends - Focusing on the Right Test Tools
and Industrializing Their Usage

March 11, 2015
Copyright 2015 Vivit Worldwide
Brought to you by
Vivit Testing, Quality and Application Lifecycle
Management Special Interest Group (TQA-SIG)
and the South Africa Chapter
Leaders: Bernard P. Szymczak Jr, Andreas Birk, Robert Linton,
Christopher J. Scharer, Damian Versaci
www.vivit-worldwide.org
Hosted by
Bernard P. Szymczak Jr
Lead IT Systems Engineer at
Progressive Insurance
Vivit TQA-SIG Leader
Todays Presenter
Mihai Grigorescu
Accenture Global HP Alliance Testing Lead
Vivit South Africa Leader
Housekeeping
This LIVE session is being recorded
The recording will be available on BrightTALK
immediately after this session
Q&A: Please type questions in the Questions
Box below the presentation screen
Additional information available for you
behind the Attachment button and later on the
Vivit website
AGENDA
Intro Accenture testing

Testing trends what the analysts say
Test Architecture and how TMMi fits into the picture
Testing ecosystem, choosing the right tools sets
ALM industrialization
Q&A
SAP Test tools implementation and support partner

HP SaaS SAP Sol Man to QC integration partner
Fig.1
Source: IDC, Doc #249925 / July 2014 - Worldwide and U.S.
Discrete Testing Services 20142018 Forecast
Fig.2
Source: HP/SOGETI, World Quality Report 2014-2015
Testing Spend
Worldwide Discrete (Standalone) Enterprise Application Testing Services market was $16.7B in 2013 and growing 15% annually to
$33.7B in 2018 (Fig.1)
U.S. Discrete (Standalone) Enterprise Application Testing Services market was $8B in 2013 and growing 13% annually to $15B in
2018 (see IDC report for graph)
Budget spend % per projects is continuously increasing (Fig.2)

Drive down costs
Support for complex infrastructure testing
(compatibility, interoperability)
Drive productivity gains (e.g. more time for
staff to pursue other initiatives)
Test staff augmentation
Standardize testing tools and processes
Leverage third-party tool expertise, talent and
skills
Automate regression testing (to support
complex business processes across multiple
Help with test data management services
Standardize internal QA environment across
projects
As insurance against potential liability resulting
from insufficiently tested products by in-
Leverage fully managed application lifecycle
testing services
Leverage fully managed specialized
application test frameworks
Support for localization – country
specific readiness testing
Use turn-key test solutions for performance
test or test automation
Support for vertical solution testing (e.g.,
banking, telecom)
Drivers for using 3rd party services

While drivers are multi-faceted, 31% of all respondents
indicate that the primary driver for using 3rd party testing
services centers on driving down costs.
37% of public sector organizations indicate the primary
driver for using 3rd party testing services is support for
complex infrastructure testing, with the second most
important being driving down costs.
Other
10
15
20
25
30
35
Source: IDC, Doc #249925 / July 2014 - Worldwide and U.S. Discrete Testing Services 20142018
Forecast
The top three characteristics buyers indicated were most

significant when selecting a provider for application testing
services were: reasonable costs (35%), high level of
responsiveness/rapid deployment (20%) and cloud testing
experience (20%)
Fig.1
Fig.2
Challenges and Outsourcing trends

There is common trend of outsourcing testing services to specialized vendors, with a drop from 51% of companies using
internal resources in 2012 to 30% in 2014 (Fig.1)

In Agile environments, hybrids persist (i.e. Scrumfall), no real Agile environments, no specific methodologies nor adequate
tools utilized, with difficulties in executing automation (Fig.2)
There is an increase in app

delivery which will reach
approximately 120 releases
per annum per app by 2020
Companies are already
planning for upgrading to
HP ALM 12.x
Source: HP, March 2014
Source: VIVIT webcast poll, June 2014
Source: HP, SOGETI - World Quality Report, 2012-2013
The main testing drivers for companies are: reduced time to

market, reduced costs, better resource management
(Fig. 1)
Companies are investing in TaaS (Testing as a Service),
centralization, as well as managed services and outsourcing
their testing capabilities (Fig. 2)
Conclusion: we believe it is paramount to ensure

Source: HP, SOGETI - World Quality Report, 2013 -2014
that test tools are optimized / industrialized within any test

organization in order to support the main testing drivers and
their respective processes adequately.

Conclusions key take aways
Digital transformation / business is here. Ones QA methodologies and tools need to cater
digital/TaaS/mobile testing. Virtualization (network, services etc.), testing in the cloud are all
elements to consider and invest in.
Applications are more fluid, delivered faster, which implies, leaner development & testing.
Agile will take even more precedence than before, consequently develop competencies and
centralized capabilities.
More automation (non-traditional) + DevOps are required for faster, organized release to
production and for bridging the gab between dev teams and operations.
With increase in overall testing spend, investing (more) in tools adoption is the next logical
inherent step.
Security testing is still undermined (Sony knows for sure), companies need to invest in proper
enterprise IT security governance frameworks and conduct more testing in this space.
Investments in skills and tools on one side, as well as change management on the other are
the crucial elements in winning the testing battle.
With commoditization of testing, one needs to invest into more creative cost efficient TaaS
models, more agile service based, output driven models, cross-shared and cross-skilled
resources vs. legacy, traditional waterfall processes, dedicated resources and restrictive
licensing models.
Test architecture and how TMMi fits into the picture

What is a Test Architect?
What a test architect IS
Testing expert: a test architect MUST come from a testing background, with experience in all
delivery models
Advisor: solution architecture, estimations, test planning, metrics definition (and
implementation), sales proposals
Fixer: get into challenged projects, produce short term/quick wins and long term improvement
plans
Testing evangelist: deliver papers, presentations, training, provide technical leadership and
strategic direction for their testing organization
Implementer: Setup of TCoE / Test Factory, execute Test Assessments (Internal, External &
TMMi), Enterprise Test Strategy (Definition/Setup), Test Optimization & Improvement, NonFunctional Testing, Standard and Emerging Technologies / Test Tools
Great article on the topic here:

https://blogs.oracle.com/johnmorrison/entry/test_architect
What a test architect ISNT

Test Manager
Fire-fighter for daily
operational issues
Delivery person (project
wise)

How TMMi fits in the picture of testing tools
Test Maturity Model
integration
(TMMi)
Test Capability Models

(based on TMMi)
Test Tools Maturity

Model
(based on TMMi)

TMMi has 5 levels
A TCoE will aim at a level 2 to 3, while test factories, between level 3 and 5
Out of experience, we know that the perceived maturity level is higher than the actual one
POLL 1
POLL 2

Accenture based
Test capabilities models
(based on TMMi)

Defect
Management
Test
Management
Requirement
Management
Advanced
reporting
(Project
dashboards)
Project
templates
KPI Scorecard
Some
automation &
performance
Level 3
Defect
Management
Test
Management
Basic reporting
Level 2
Ad-hoc usage
Basic defect
management
Level 1
Defect
Management
Test
Management
Requirement
Management
Planning
(release
management)
Automation,
Performance
Basic Mobile,
Security
Advanced
reporting
(Dashboards
across
programs)
Scorecards
Tool QA
governance
Basic CI
Low
Defect
Management
Test
Management
Requirement
Management
Planning
(release
management)
Libraries and
cross project
asset sharing
and reuse
Automation,
Performance,
Mobile, Security
Complete
reporting
capabilities
Executive
Scorecards
Tool QA
governance
Seamless
integration to 3rd
party tools (CI)
High
Testing Tool Usage- Capability Maturity Level
Level 5
Level 4
Testing ecosystem and focusing on the right tools
Define (Test Architect) what tools to use in

the test organization based on needs and
long term requirements
Ensure that all testing tools are integrated

for a streamlined testing process
Maximize tools ROI via:

a) Tools industrialization
b) Strategic partnerships with tools
vendors, for band discounts
POLL 3
POLL 4
POLL 5
POLL 6
Test Tool
Map
Tools
Strategy
Shift Left
Quick Wins
Tools Team
HP ALM
Industrialization
Upgrade
versions
Reporting
Upgrade
editions
Integrations
Accelerators
Test Tool
Map
Set up a test tools roadmap vs. your value chain / Software Development Life Cycle model
Ensure that all testing areas are covered (Manual, Automation, Performance, Mobile, Sec., TDM, TEM
Tools
Strategy
Define and implement an overall test tool strategy for the next 3-5 years
The test tool strategy needs to include capital and operating expenses (including software, hardware and
services costs) in order to provide the most efficient, cost effective solution for the organization
Test tools strategy: The enterprise wide approach

which describes the solution architecture for the
deployment of all integrated test tools suites:
manual, automation, performance, mobile &
security, incl. additional tools such as requirements
accelerators, COTS and in-house grown (internal)
software utilized in the testing effort.
Tools Team
Set up a dedicated test tools team to handle all test technologies in the organization
Establish a test tool service catalogue & use the tools as bridging mechanisms into other business areas
Upgrade
versions
Upgrade to the latest HP QC/ALM versions to make sure your organization gets the best value out of the
tool
Make sure that your current QC version is still under support
Version
Support End Date
HP ALM / QC Vendor Support Time-Table

9.x
10.x
11.0
Nov 2010
Aug 2013
Jan 2015
ALM/QC 11.0 will be out of support in Jan 2015

(latest communication from HP):
11.5
TBA
11.52
TBA
12.0
TBA
12.01
TBA
What extended support entails:

Clients can still buy support beyond the standard EOL (End of Life)
product, e.g. 31 Jan 2015 in this case, however:
It requires a CSP Customer Support Plan, for max of 1 year @
150% of list price
Availability of support resources: best effort (when available)
Support type: no product fixes/enhancements requests etc.
Why consider upgrading editions:
Avoidance of extended maintenance costs

Continuation of vendor support after QC/ALM 11.0 end of support (Jan 2015)
Decreased risks of troubleshooting issues and receiving product fixes/patches etc.
License assignment per domain/business area (for all versions >11.5), enabling the possibility of producing a charge back model (to
business) for recuperating software license costs (best suitable for medium to large installations/test capabilities)
Less disruption: no additional required upgrades (from older versions to more recent ones) in the next 3-4 years due to end of support.
Access to newest ALM features & functionalities
Upgrade
editions
If not already on Premier, consider the upgrade in Editions, in order to maximize the ROI on the test tools
Accelerators
Utilize accelerators, either tools or other reusable assets in order to achieve maximum efficiency with the
organization test tools sets.
SAP ERP QC Project Templates, Predictive Analytics, E-approve, Business Intelligence, Operations
accelerator, Requirements Management aiding tools, SAP/Automation accelerators etc.
Integrations
Ensure that there are proper integration flows between test tools and that redundant / duplicate work is
not executed.
Examples: AGM <-> ALM, Jira <-> ALM, Sol Man <-> ALM, ALM <-> PPM/XS, TFS <->ALM etc.
Reporting
Define and implement an adequate test metrics framework

Use ALMs innate reporting capabilities (document generator, graphs and dashboards, KPI scorecards,
cross project reporting), Business Views, in conjunction with 3rd parties business intelligence apps;
publish QC/ALM graphs to Share Point Site (SPS)
Quick Wins
Use the test capability to produce quick wins list that can be used with immediate effect in order to
improve the test processes (via the tools)
Example below: defect management process, consistent project template
Training: ART (Adoption Readiness Tool, other practical hands-on classes.
Shift Left
Use ALM to drive Shift Left transformation into the organization

Use simplified version of Shift Left (from a defect mgmt. perspective) improve quality via defect root
cause analysis and improved management
Shift Left
Pushing quality upstream in the SDLC by enabling the testing capability with methodologies and tools
Knowledge sharing capability development (training, coaching)
Earlier defect detection, inspections, walkthroughs, JADs
Methodologies + Tools
Q&A
MIHAI.GRIGORESCU@ACCENTURE.COM
Upcoming Vivit SIG Webinar

Best Practices in IT Architecture
March 18, 2015 at 10:00 am EDT / 15:00 pm CET
Brought to you by the
Vivit Business Service Management Special Interest Group
For more information and registration go to:

Vivit Board of Directors election

Vote now on the
2015 Vivit Board of Directors election
and the proposed Bylaw changes:
http://www.vivit-worldwide.org/surveys/?id=459388
HP Discover Las Vegas 2015

June 2 4, 2015 at
The Venetian Resort
in Las Vegas.
All members can

Register Now via the
Vivit link
www.hp.com/go/discover/vivit
to save $300.
Vivits Deep Dive Sessions
Deep Dive Sessions will be offered on Monday,

June 1st from 1:00 5:00 pm. Find more
information on website under the training section.
Thank you
Copyright 2015
2015Vivit
VivitWorldwide
Worldwide

2015.03.11 Testing Trends Slide Deck EMEA

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

2015.03.11 Testing Trends Slide Deck EMEA

Hochgeladen von

Copyright:

Verfügbare Formate

Testing Trends - Focusing on the Right Test Tools

and Industrializing Their Usage

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide

Intro Accenture testing

Copyright 2015 Vivit Worldwide

Intro Accenture testing

Copyright 2015 Vivit Worldwide

SAP Test tools implementation and support partner

Intro Accenture testing

Copyright 2015 Vivit Worldwide

Testing trends what the analysts say

Copyright 2015 Vivit Worldwide

Testing trends what the analysts say

Drivers for using 3rd party services

Copyright 2015 Vivit Worldwide

The top three characteristics buyers indicated were most

Testing trends what the analysts say

Challenges and Outsourcing trends

internal resources in 2012 to 30% in 2014 (Fig.1)

tools utilized, with difficulties in executing automation (Fig.2)

Copyright 2015 Vivit Worldwide

Testing trends what the analysts say

There is an increase in app

Source: HP, March 2014

Copyright 2015 Vivit Worldwide

Source: VIVIT webcast poll, June 2014

Testing trends what the analysts say

Source: HP, SOGETI - World Quality Report, 2012-2013

The main testing drivers for companies are: reduced time to

Conclusion: we believe it is paramount to ensure

Copyright 2015 Vivit Worldwide

that test tools are optimized / industrialized within any test

Testing trends what the analysts say

Test architecture and how TMMi fits into the picture

Great article on the topic here:

Copyright 2015 Vivit Worldwide

What a test architect ISNT

Test architecture and how TMMi fits into the picture

Test Capability Models

Test Tools Maturity

Copyright 2015 Vivit Worldwide

Test architecture and how TMMi fits into the picture

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide

Test architecture and how TMMi fits into the picture

Copyright 2015 Vivit Worldwide

Test architecture and how TMMi fits into the picture

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide

How TMMi fits in the picture of testing tools

Test architecture and how TMMi fits into the picture

Testing ecosystem and focusing on the right tools

Define (Test Architect) what tools to use in

Ensure that all testing tools are integrated

Maximize tools ROI via:

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide

Copyright 2015 Vivit Worldwide