Test Suite

Knowledge sharing - Solution Manager 7.2

By: Endre Rasmussen

At: Coop Norge
Date: 24.10.17
Expectations
“Experience Sharing and Demonstration of Test Suite,
Solution Manager 7.2 by Coop”
Coop will share their experience of using Test Suite in Solution Manager 7.2. Learn and
understand how Solution Documentation is the foundation to successful use of Test
Suite. Gain insight and build experience in test planning, reporting, execution and defect
handling with integration to ChaRM.
Agenda

• Facts about Coop

• SolMan usage in Coop
– High level concept
• Solution Documentation
– Experience and learning points
• Test Suite
– Experience and learning points
• Demo
 Facts about Coop
28.000
29,4 % 45 billion employees
marked share in in turnover
grocery
1350
stores 1,5 million
members

SAP landscape
• ECC Retail, ECC Industry, ECC slag, SCM, CRM, BW (HANA/ABAP), CAR
(HANA/ABAP), SLT, Enterprise Portal, NWDI
• Solution Manager 7.2 SP4
• Three tier system landscape + sandbox
• >4000 users

Data from March 2017

 High-level concept in Coop
One Solution with integration to other modules
Business Process Operation
• Business Process Monitoring (BPMon)
Test Suite • Job Monitoring
• Link to both manual and automated (CBTA) • Data Volume Management
test scripts
• Business Process Change Analyzer (BPCA)
• Scope and Effort Analyzer (SEA) - pilot

Custom Code Management

• Code Inspector (integration to ChaRM)
• CCLM – Strategic roadmap in progress
Change Control Management
• Integration to ChaRM documents
• Process structure and document
change control
Business Process Improvement
• Measure ROI based on usages statistic
in the production system
• Identify critical business processes
based on most used objects
• Identify improvement areas based on
not used objects (lack of training etc.)
• Create Solution reports for audit and
new projects
Solution Documentation
 Solution Documentation
Our «Single Source of Truth»

• Solution: 1 Solution, 75 logical component groups, 2 process groups

• E2E Processes: 5 Scenarios, 9 Processes, 159 Process steps, 7 diagrams
• Modular Processes: 144 Scenarios, 458 Processes, 2395 process steps, 214 diagrams,
4385 test cases (only 1500 assigned to processes)

Statement: Process modelling is the foundation for control and measure of our solution, and
a common visual language between IT and Business
 Solution Documentation
Coop Experience and learning points

General recommendations
- Recommend to upgrade to SP05 with possibility to define business process variants
- Identify the processes responsible to ensure ownership and continues maintenance
- Recommend numbering the defined processes as part of naming standard, for easier
reporting and search
- Define needed customer attributes from start to simplify the search and reporting (via
SOLMAN_SETUP)
- Use collaboration diagram in order to allow modelling of interfaces to third parties
- Define how the process can be measured and controlled (input to BPMon)

General experience
- Very good search and filtering functionality
- Experience that IE browser is slow to work with, Chrome is better but does not have the
edit online function for document handling
- Not possible to mass update (example attributes). Mass update comes in SP5 but not
stated what its used for
 Solution Documentation
Coop Experience and learning points

Relevant for Test Suite

- Decide whether to create and assign test cases on step or process level
- Create a manual test case template for consistency (expected result, actual result etc)
- Recommend to ensure test cases reflects the modelled processes, including interfaces
- Recommend to assign custom attributes to test cases, for easy test planning (example:
test phase relevance, criticality, relevant for test automation)
- Note that test cases not possible to be created for interfaces in interfaces library. This
makes it hard to report and sign off testing of critical interfaces

Test Case reflects

Modell process Create test case based on template the process modell
Test Suite
 Test Suite
Experience taken from SAP upgrade project in Coop

• Scope: Upgrade all SAP systems (3xECC, BW, CRM, SCM, Portal, TREX, Java Hub, NDWI)
• Timeline: 6 months (April – October 2017)
• Facts: 38 systems upgraded, 130 involved people, 1766 manual test cases executed, 159
errors detected and corrected

TEST FASE PACKAGES TEST CASES NO IN PROGRESS ERRORS OK DEFECTS COMPLETION (%)
RESULT
SANDBOX (ST) 124 503 16 4 16 467 36 93

DEVELOPMENT (DT) 9 89 21 5 2 61 2 69

USER ACCEPTANCE TESTING 251 931 0 0 0 931 119 100

(UAT)
PRODUCTION VERIFICATION 65 308 0 0 1 307 2 100
(PRODT)
TOTAL OF ALL TEST PHASES 449 1831 37 9 19 1766 159 100

One of the key success factors was the preparation work, identify and secure test
coverage of critical business processes and related interfaces
 Test Suite
High level Test flow
Test
Manager
• Run BPCA to scope test
• Creation of Test Plan, Test
Package and Test Case and
Tester assignment
• Creation of Test Reports
Tester
• Test Execution via Tester
Worklist
• Test Note creation
• Test Defect creation
Developer
• Test defect handling in ITSM
• Using Defect Correction in
ChaRM (close link to test defect)
 Test Suite - Test Planning
Test Experience and Learning points
Manager

Experience
• Nice integration with Solution Documentation, structure is maintained and well known
• Note that changes in test case (SolDoc) will not be visible in existing test packages, new test
package needs to be created and assigned to tester
• Missing mass update function of tester in test packages for Test Manager. Coming in SP05
• Good reporting possibilities (drill down in all levels)
• No experience with BPCA in 7.2 (not working for Maintenance Planner during project period)
• No experience in Test Case sequence, possible but not used
Learning points
• Decide on which level in structure the test plan and test packages should be created for easy
reporting and follow up

Example of report:
 Test Suite - Test Execution
Experience and Learning points
Tester

Experience
• Tester recognize the structure in tester worklist from the process structure, which is good
• Too many pop ups/different windows when working with manual execution. Solved with Focus Build
• Test Cases opens in a separate window, not embedded as in 7.1. Solved with Focus Build
• Many refresh buttons under Tester Worklist. Needs to be refreshed manually
• Start Test Execution button makes sense if you have test case on process step level (Coops
decision to have test case on process level). Solved with Focus Build
• No experience in TBOM recording, decided not to use refer to statement above
Learning points
• Decide the use of Test Note and Test Result, and method (check in check out or upload new
version)
• Evaluate the available Focused Build solutions offered by SAP related to Test Suite (Simplified test
execution etc)

NEWS: Focused Build for 7.2 - New manual test solution with grid approach fully
embedded in Test Suite etc – link (slide 27-45)
 Test Suite - Defect Handling
Experience and Learning points
Developer

Experience
• Nice follow up document integration between Test Defect (Incident) and Defect Correction
(ChaRM), text and field entries can be copied over
• No interlink when processing Test Defect and Defect Correction (Note: Coming). Developer
needs to work on both Test Defect and Defect Correction document when updating status etc
• No direct link to test notes from Test Defect, only link to test case (where actual result is blank)
• Note that Testers must go into ITSM in order to reply and confirm reported Test Defects (direct
link from tester worklist or mail notification)
Learning points
• Analyse the standard Defect Handling workflow and evaluate if customizing is needed

COMING: Status profiles for Test Defect and Defect Correction interlinked – link (slide 24)
Demo
SolMan 7.2 SP4