The Company….

♦ The Market Leader in Globalization

Technology
– Pioneers in visual translation environments
– Solutions for major platforms & programming
languages
– 24.9% owned by Corel Corporation
– Sales growing 40+% annually
– Currently 9,000+ customers worldwide
– Offices in Ireland and USA
• Japanese office to open Q3 2003
 Challenges Simultaneous Translation,
Engineering & Testing

♦ We live in a Parallel Universe

– Translation
• Seldom starts or finishes on time
• Difficulty getting source materials from development teams
• Diverse goals of translation/development teams
• Goal : Translate as much as possible as quickly as possible
- Quality Assurance
• To monitor quality levels of Engineering
• To monitor quality levels of Translation
• Goal : To locate, verify and report as many quality
performance issues during the schedule
– Engineering
• Struggle to control inputs to drive definite outcomes
• Create continuous streaming of localization materials to
Translators and Quality Assurance team
• Goal : To locate, debug and fix as many
quality performance issues during the schedule
 Challenges Simultaneous Translation,
Engineering & Testing

♦ We live in a Parallel Universe

– Translation
• Seldom starts or finishes on time
• Difficulty getting source materials from development teams
• Diverse goals of translation/development teams
• Goal : Translate as much as possible as quickly as possible
- Quality Assurance
• To monitor quality levels of Engineering
• To monitor quality levels of Translation
• Goal : To locate, verify and report as many quality
performance issues during the schedule
– Engineering
• Struggle to control inputs to drive definite outcomes
• Create continuous streaming of localization materials to
Translators and Quality Assurance team
• Goal : To locate, debug and fix as many
quality performance issues during the schedule
QA Engineering Overview of
Test Phases

– Evaluation & Planning Phase

– Goal : Defines the scope of the Test Plan
– Includes Resource Plan & Scehdule
– Product Modularization Report & Baseline Defined
- Exploratory Phase
- Goal : Gain familiarity with application
- Pretesting of application
- Development Phase
- Goal : Create Test Scenarios to support Test Plan
- Execution Phase
- Modular testing of application areas
- Internationalization & Language Verification
- Pruning Phase
- Interlaced throughout your testing cycles
- Provides audit for improving process and test scenarios
- Regression & Verification Phase
- Retest and verify Engineering activities
- Sign-off Phase
- Additional testing requirements for release and mastering
QA Engineering Designing a
Test Case

Clearly Defined Goal

* What is the function of the test ?

Clear Test Activity

* What steps do I follow to create the
test scenario ?

Deterministic Outcome
* How do I determine success/failure
Remember!
of test ?

Tests can only prove that errors exist,

not that a product is free of errors!
QA Engineering Types of
Testing

– Common Types of
Tests
– MenuWalker
– Functional
- Scenario
- Random
- Automated
- Interoperability
- Language Quality
- Data Path
- Boundary
Incremental Testing Strategy
- I18N
For optimal efficiency, your L10N teams should not
test baseline functionality! It should focus on
incrementally testing that the product integrity is
maintained through the localization process.
QA Engineering Types of
Testing

– Common Types of MenuWalker Tests

Tests
– MenuWalker Objective

– Functional • Exercise all UI of a

translated product
- Scenario
• Locate all cosmetic issues
- Random Execution
- Automated • Easily Automated
- System • Manual Scripts easy to
- Acceptance develop

- Language Quality • Various Tools : Alchemy

CATALYST RTV, ToolProof,
- Data Path Visual Test
- Boundary Deterministic Outcomes
- I18N • UI devoid of all cosmetic
and layout bugs
- Performance
• Translation Completeness
QA Engineering Types of
Testing

– Common Types of Functional Tests

Tests
– MenuWalker Objective

– Functional • Verify the functional

integrity of application
- Scenario
Execution
- Random • Modular approach is best
- Automated • Focus on Unit, Module and
- System Components

- Acceptance • Data Comparative cases

are easily automated
- Language Quality
Deterministic Outcomes
- Data Path
•Product is functionally
- Boundary consistent with original base
language application
- I18N
- Performance
QA Engineering Types of
Testing

– Common Types of Scenario Tests

Tests
– MenuWalker Objective

– Functional • Verify product functional

integrity in various real-
- Scenario world test scenarios
- Random Execution
- Automated • Previous regression tests
and technical support
- System feedback can be useful
- Acceptance • Can be complex to
- Language Quality automate

- Data Path • Focus on many System

variations
- Boundary
Deterministic Outcomes
- I18N •Product is functionally
- Performance consistent with original base
language application
QA Engineering Types of
Testing

– Common Types of Random Tests

Tests
– MenuWalker Objective

– Functional • Verify product functional

integrity using random test
- Scenario data inputs and scenarios
- Random Execution
- Automated • Unstructured testing of
application in real-world
- System scenarios
- Acceptance • Almost impossible to
- Language Quality automate

- Data Path • Only useful with

experienced QA Engineers
- Boundary
Deterministic Outcomes
- I18N • Generally none, but may
- Performance prove ‘Product Robustness’
QA Engineering Types of
Testing

– Common Types of Automated Tests

Tests
– MenuWalker Objective

– Functional • Verify product functional

integrity using automated
- Scenario test cases
- Random Execution
- Automated • Structured testing of
application in controlled
- System scenarios
- Acceptance • Very effective with
- Language Quality comparative test cases

- Data Path Deterministic Outcomes

- Boundary • Software integrity is

maintained in controlled test
- I18N cases
- Performance • Acceptance of software at
significant project
milestones
QA Engineering Types of
Testing

– Common Types of Automated Tests

Tests
– MenuWalker Objective

– Functional
Automator, beware ! • Verify product functional
integrity using automated
- Scenario test cases
- Random
Automation is Resource Intensive Execution
Automated
- •Developing Automated Scripts can be time-consuming
• Structured testing of
application in controlled
Need understanding of programming techniques
- •System scenarios
• If not developed correctly, can be very high on maintainence
- Acceptance • Very effective with
No Replacement for real testing
- Language Quality comparative test cases
• Can compliment but not replace real ‘testers’ Outcomes
- Data Path Deterministic
Not everything adapts well to test automation
- Boundary • Software integrity is
• Good : Tests that verify an applicationmaintained
works in a in controlled
particular test
way
- I18N cases
• Bad : Tests that find previously undiscovered errors
- Performance • Acceptance of software at
significant project
milestones
QA Engineering Types of
Testing

– Common Types of System Tests

Tests
– MenuWalker Objective

– Functional • Verify product functional

integrity under different
- Scenario system environments
- Random Execution
- Automated • Structured test cases
using various OS
- System
• Lends itself to automation
- Acceptance
Deterministic Outcomes
- Language Quality
• Functional completness on
- Data Path various OS environments
- Boundary • Performance of application
- I18N • Verification of product
license integrity
- Performance
•Completeness of Product
International Support
QA Engineering Types of
Testing

– Common Types of Acceptance Tests

Tests
– MenuWalker Objective

– Functional • Verify minimum product

quality to commence QA
- Scenario cycle
- Random Execution
- Automated • Very Structured test
cases
- System
• Improve by including
- Acceptance previously regressed SPRs
- Language Quality • Significant automation
- Data Path canidate

- Boundary Deterministic Outcomes

• Verification of minimum
- I18N functional completeness
- Performance • First line of defence for
detecting ‘corrupted’ builds
QA Engineering Types of
Testing

– Common Types of Language Quality Tests

Tests
– MenuWalker Objective

– Functional • Verify translation is

consistent with linguistic
- Scenario guidelines
- Random Execution
- Automated • Some fundamental tests
can be easily automated
- System
• Sampling generally creates
- Acceptance the best results due to
- Language Quality volumes

- Data Path Deterministic Outcomes

- Boundary • Translation is consistent

with Corporate Terminology
- I18N guidelines across component
boundaries
- Performance
• No cosmetic/layout issues
exist (e.g. Spelling errors)
QA Engineering Types of
Testing

– Common Types of Language Quality Tests

Tests
– MenuWalker Objective

– Functional • Verify translation is

Examples of Language Quality Automation consistent with linguistic
- Scenario guidelines
- Random
Translation Consistency & Completeness Check
Execution
•Relatively easy to automate
- Automated • Some fundamental tests
•Should always be part of an acceptancecantest
be easily automated
- System
Spell Checking • Sampling generally creates
- Acceptance the best results due to
• Relatively easy to automate
- Language Quality volumes
• Should always be part of an acceptance test Outcomes
- Data Path Deterministic
Duplicate Hotkeys/Layout Issues/Placeholders
- Boundary • Translation is consistent
• Relatively easy to automate & locate with Corporate Terminology
- I18N guidelines across component
boundaries
- Performance
• No cosmetic/layout issues
exist (e.g. Spelling errors)
QA Engineering Types of
Testing

– Common Types of Data Path Tests

Tests
– MenuWalker Objective

– Functional •Verify data integrity of

application based on
- Scenario knowledge of internal
workings of application
- Random
Execution
- Automated
• Structured data used to
- System test application
- Acceptance • Focus on inputs driving
- Language Quality deterministic outputs

- Data Path • Easy to automate with

comparative data outcomes
- Boundary
Deterministic Outcomes
- I18N • Data input and generated
- Performance outputs are consistent with
base language application

NOTE : Sometimes referred to as Grey box testing

QA Engineering Types of
Testing

– Common Types of Boundary Tests

Tests
– MenuWalker Objective

– Functional •Verify application

functionality with extreme
- Scenario data inputs
- Random Execution
- Automated • Structured data used to
test application
- System
• Relatively easy to
- Acceptance automate with comparative
- Language Quality data outcomes

- Data Path Deterministic Outcomes

- Boundary • Data input and generated

outputs are consistent with
- I18N base language application
- Performance • Applicatio robustness –
illegal data values do not
crash application
QA Engineering Types of
Testing

– Common Types of L18N Tests

Tests
– MenuWalker Objective

– Functional • Verify application supports

all Locales
- Scenario
Execution
- Random • Structured data that
- Automated focus on locale used
- System • Relatively easy to
automate with comparative
- Acceptance data outcomes
- Language Quality • Break down into categories
- Data Path : Script & Regional

- Boundary Deterministic Outcomes

• Application supports local
- I18N script and regional
- Performance conventions
QA Engineering Types of
Testing

– Common Types of L18N Tests

Tests
– MenuWalker Objective

– Functional • Verify application supports

Structure of L18N Tests all Locales
- Scenario
Execution
- Random
Language/Script
Locale
• Structured data that
• Collation
- Automated
• Casing
focus on locale used
Language/Script
- System
• Character Set (SBC, DBCS, BIDI) • Relatively easy to
Regional
automate with comparative
- Acceptance
• Inputs – Keyboard, IME
data outcomes
- Language
• Output – FontsQuality
Display/Printing
• Break down into categories
• Character Conversions
- Data Path : Script & Regional
Regional
- Boundary Deterministic Outcomes
• Number, Currency, Time, Date, List Seperator,
• Application supports local
- I18N
Address Format, Telephone Format
script and regional
- Performance conventions
QA Engineering Types of
Testing

– Common Types of Performance Tests

Tests
– MenuWalker Objective

– Functional •Determine how system

operates under ‘Load’
- Scenario conditions
- Random Execution
- Automated • Structured environments
used to test application
- System
• Relatively easy to
- Acceptance automate
- Language Quality Deterministic Outcomes
- Data Path • Application operates
- Boundary successfully under stressful
load conditions
- I18N • May identify areas of
- Performance functionality for
optimization by Development
team
QA Engineering Types of
Testing

– Common Types of Performance Tests

Tests
– MenuWalker Objective

– Functional
Factors to Consider during Performance •Determine how system
Testing
operates under ‘Load’
- Scenario
Network Capacity conditions
- •Random
Concurrent users Execution
- •Automated
Remote Sessions/Connections • Structured environments
used to test application
- System
Disk, Memory and CPU Usage
• Relatively easy to
- •Acceptance
Run system under minimum and maximum boundary conditions
automate
- Language
Transaction LoadingQuality Deterministic Outcomes
- •Data Path
Continuous transaction loading over time
• Application operates
- Boundary successfully under stressful
load conditions
- I18N • May identify areas of
- Performance functionality for
optimization by Development
team
QA Engineering Managing &
Tracking

– Tracking your QA
Engineering Projects
– Coverage Reports
– By Platforms
– By Module,
Component, Unit
– By Tester
– By Build, Release
– By Language
– Bug Tracking
– 3 Point Tracking
450
400

– By Category,
350
300
Open

Severity
250
Closed
200
Pending
150

– Find vrs Fix Ratio 100

50

– Pending Tracking
-
1 2 3 4 5 6 7 8

– Language Quality
Reports
– Terminology
Consistency Report
– Sampling Reports
– Error Density
 QA Engineering Tools to Help
you Manage

– Alchemy CATALYST 4.0

– RTV Component
– Locates and tracks common
localization issues
– Validate Expert
– Automates detection and
tracking of localization issues
– SDL ToolProof
– Identifies UI Issues
– General Automation Tools
– Seque ( Tester)
– SilkTest International
– SilkPerformer
– SilkRadar
– Rational Systems
– Wide selection of Developer
Tools
– Code Analysis
– WEB Services Analysis
– Mercury Interactive
– WinRunner
– Test Director 7.x
QA Engineering When to start
IQA

Traditional Product Development Cycle

Base Product Development

Base Product QA Cycle

Localization Project Cycle

IQA Cycle

Disadvantages
–Localization & IQA not integral to base development
–Base Product completes before any focus on International Issues
–IQA bugs only addressed after release of base product
–No sharing and coupling of test plans between Base QA and IQA teams
–No International Product Development Planning

–IQA Starts after Localization begins

–Translated product required to start IQA effort
–Any IQA issue that needs to be fixed is eventually addressed post-base produuct
release
QA Engineering When to start
IQA

Ideal Product Development Cycle

Base Product Development

Product QA and IQA Cycle

Localization Project Cycle

IQA Cycle

Advantages
–IQA is an integral part of the PD Lifecycle
–Releasing localized products is a Corporate Wide Goal
–Base QA team are aware of and implementing IQA in their Base Product QA
plans
–Development fix IQA issues as they develop Base product
–Corporate Goal : Universal English Product

–Localization Cycle is parallel to development effort

–Localization – Start as late as possible to finish as early as possible
–Simultaneous Release of language products now feasible!
Thank you….
After Lunch Challenge

Devise a test plan for the following code segment…

integer foo(integer x, integer y, integer z )

{
integer Result

let Result = x * y / z

return Result
}