ROI and the Value of Machine Translation

Presented by Nicholas McMahon

Moderated by Paula Shannon
July 15, 2009
 Introductions

Nicholas McMahon
VP, Global Technology Solutions, Lionbridge
Nicholas.McMahon@lionbridge.com
Over 13 years experience working with some of the world’s most
successful global organizations
Provided strategic and tactical global leadership as a foundational
executive for two US divisions of European companies, as well as a
VAR business

Paula Shannon
CSO, SVP and General Manager, Lionbridge
Paula.Shannon@lionbridge.com
More than 23 years experience in the translation and localization
industry
Responsible for sites in 26+ countries, driving new services and
sustainable solutions, ensuring the continued delivery of innovation
and execution excellence to a broad range of Global 1000 customers

Company confidential – distribution prohibited without permission

 About Lionbridge

Global Scale Market Leadership

4,600 employees 26 countries Leader in $14B services industry
• Global network of 25,000 translators • Translation and adaptation of products
and content for international markets

Hosted Technology Global Clients

Web-based language technology Recurring relationships with 500+
platform global clients
• Enhances competitive advantage • 80% of revenue comes from recurring clients
• Drives efficiency • 12 of the Fortune 20 companies are client

Company confidential – distribution prohibited without permission

 Agenda

Basic MT Introduction
3 Basic Considerations for MT Evaluation
Top 3 Applications of MT
Enterprise Cost Considerations for MT
Business considerations and ROI Modeling
Live Q&A

Company confidential – distribution prohibited without permission

 Basic MT Introduction

Company confidential – distribution prohibited without permission

 Three Types Of Machine Translation

Rules Based
iTranslator
Statistical
Google
Hybrid
Systran Enterprise
Server 7
Company confidential – distribution prohibited without permission
 Basic MT Introduction

Company confidential – distribution prohibited without permission

 Agenda

3 Basic Considerations for MT Evaluation

Company confidential – distribution prohibited without permission

 Top Three Considerations for MT

MT performance in one
language is NOT MT
performance in another
language or even in another
language direction

Company confidential – distribution prohibited without permission

 Top Three Considerations for MT

MT is not an
“every language solution”
for any of the MT Engines

Company confidential – distribution prohibited without permission

 Top Three Considerations for MT

MT at an Enterprise Level
is NOT free

Company confidential – distribution prohibited without permission

 Agenda

Top 3 Applications of MT

Company confidential – distribution prohibited without permission

 Top Three Applications of MT:
Chat

Company confidential – distribution prohibited without permission

 Top Three Applications of MT:
Chat
MT Chat Approaches
Human to human
instant dialogue
Human to semi auto
response based on
key word findings

Evolution
Early adoption – Value focus, very little distinct ‘ROI’
modeling

Company confidential – distribution prohibited without permission

 Top Three Applications of MT:
Support Content

Company confidential – distribution prohibited without permission

 Top Three Applications of MT:
Support Content
FAQ MT Approaches
Raw MT output for un-
translated content
Tiered Quality approach

Evolution
Volume Success and reasonable adoption
“Self serve” & “New content” increases user patience

Company confidential – distribution prohibited without permission

 Top Three Applications of MT:
User Guides

Company confidential – distribution prohibited without permission

 Top 3 Applications of MT

Approaches
TM, MT + Post Editing of content

Considerations
Early Adoption – strong indicators of
success
Goal is to MT to be ‘invisible’ in
process

Company confidential – distribution prohibited without permission

 POLLING QUESTION

Are you currently considering or using MT in any of these

situations?

Considering MT for Chat

Considering MT for FAQs
Considering MT for Volume User Content
Using MT for Chat
Using MT for FAQs
Using MT for Volume User Content

Company confidential – distribution prohibited without permission

 Agenda

Enterprise Cost Considerations for MT

Company confidential – distribution prohibited without permission

 Enterprise Cost Considerations

“Start up Costs” Costs

Data Identification/ Corpus creation

$0k - $100k

Customization or training of content

$5k - $250k

Licensing fees $0k - $200k

Integration fees $10k - $200k

Company confidential – distribution prohibited without permission

 Enterprise Cost considerations

“Post Editing” Costs

Training of resources for post editing $5k

- $10k

Post editing rates 40% – 65% of word rate

approx 10 cents – 16 cents per word

Ongoing customization/ corpus

management annually $5k - $200k

Company confidential – distribution prohibited without permission

 Agenda

Business considerations and ROI Modeling

Company confidential – distribution prohibited without permission

 ROI modeling for MT within
Instant communication
User 1 User 2

My Second figure is an animal My role of a young handsome

with a tail and four feet beau is a boy with a tail and 4
feet
What kind of meaning and nice What do you mean? A handsome
boy? boy?
My second figures looks like an My second figure is like an
animal animal
It has four feet and a tail It has four feet and a tail
Oh, I understand and am the 8th I got it, it is my 8th figure

Source: NTT Labs and Kyoto University,

Effects of Machine Translation on Collaborative Work, Dec 06

Company confidential – distribution prohibited without permission

 ROI modeling for MT within Instant communication

Establishing the value of instant communications

Ability to resolve support requests

Cost per multilingual call handling
Ability to communicate better
Value of increased time to market

Company confidential – distribution prohibited without permission

 ROI Modeling for MT within Instant Communication

MT in Chat Benefits Variables

Strong defined reference
sets produce better results
Strong defined language
use (specific to a domain
or industry)
Patience and Flexibility of
user (not for complaints!)
Favors Value of ‘currency’
of content versus quality

Company confidential – distribution prohibited without permission

 ROI modeling for MT within Instant communication

Results Defined Data Sets

5 - 25% average success rate
(resolution of issue)
User Satisfaction generally 30 –
50%

Company confidential – distribution prohibited without permission

 ROI modeling for MT within Instant communication

ROI Model – Statistical MT considered best match

10 Languages, 100,000 chat support requests

Alignment of data for Corpus Creation: $30k

Product Licenses $100k
Integration of tools $20k
Human system/ process training $10k
System tweaking : $20k

Total Cost $180k

Company confidential – distribution prohibited without permission

 ROI modeling for MT within Instant communication

Averaged Results for Calculation

10% support cases resolved
Average cost of alternate solution: $15 (FAQ routing/ human
Call)

Benefit
100,000 * 10% * $15 $150,000

Less Costs $180,000

Basic Financial ROI Year 1 -$30,000

Company confidential – distribution prohibited without permission

 POLLING QUESTION

Based on the considered MT chat model do you think the suggested

ROI is:

Very Low
Low
About Right
High
Very High

Company confidential – distribution prohibited without permission

 ROI modeling for MT applied to FAQs

Establishing the value of FAQs?

Ability of users to autonomously

resolve support requests

Increased Customer Satisfaction

Company confidential – distribution prohibited without permission

 ROI modeling for MT applied to FAQs

Benefits Variables
Strong defined reference sets
produce better results
Quality of source content
construction
Expectations of user –
disclaimers on quality
Favors un-translated content
Rules based or Hybrid best
solutions
Provision of English source links
provide additional context
Cultural expectations
Volume of source data available
Company confidential – distribution prohibited without permission
 ROI modeling for MT applied to FAQs

Results Defined Data Sets

20% - 35% average
success rate (resolution
of issue)
On average about 8%
less than English
User Satisfaction
generally 50 – 65%

Company confidential – distribution prohibited without permission

 ROI modeling for MT applied to FAQs

ROI Model – Hybrid MT Engine considered

10 Languages, 100,000 FAQs docs 450 words per doc.

Alignment of data for Corpus Creation: $50k

Product Licenses $250k
Integration of tools/ solution $60k
Rating, feedback reporting $20k
Human system/ process training $10k
System tweaking : $50k

Total Cost $390k

Company confidential – distribution prohibited without permission
 ROI modeling for MT applied to FAQs

Averaged Results for Calculation :

25% support cases resolved
Average cost of alternate solution: $35(Human Call Response)

Benefit
100,000* 25% * $35 = $875,000

Less Costs $390000

Basic Financial ROI Year 1 $485,000

Company confidential – distribution prohibited without permission

 POLLING QUESTION

Based on the considered MT FAQ model do you think the suggested

ROI is:

Very Low
Low
About Right
High
Very High

Company confidential – distribution prohibited without permission

 ROI Modeling for MT applied to User Guides

Establishing the value of MT within User Documentation

Improvement in output volume
Reduction in total attracted cost of translation
Increased user content available

Company confidential – distribution prohibited without permission

 ROI Modeling for MT applied to User Guides

Benefits Variables
Strong defined reference sets
produce better results
Quality of source content
construction
Knowledge and skills of human
resources
Systems and tools integration of
MT and TM
Target Language complexity

Company confidential – distribution prohibited without permission

 ROI Modeling for MT applied to User Guides

Results Defined Data Sets

-10% - +25% productivity
improvement
Reduction in total attracted cost
per word

Company confidential – distribution prohibited without permission

 ROI Modeling for MT applied to User Guides

ROI Model – Rules Based MT considered

4 Languages, 2,000,000 words total

Alignment of data for Corpus Creation: $5k

Product Licenses $20k
Human system/ process training $20k
System tweaking : $5k

Total Cost $50k

Company confidential – distribution prohibited without permission

 ROI Modeling for MT applied to User Guides

Averaged Results
15 % increased in productivity
Average fully loaded per word cost $0.25
(accounting for fuzzy matches)

Benefit
2,000,000 * 15%* 0.25 = $75,750

Less Costs $50,000

Basic Financial ROI Year one $25,750

(about 10% of expected costs)
Company confidential – distribution prohibited without permission
 MT ROI Take Away

Early Adoption, but MT based projects are reporting

MT solutions are dramatically improving
MT source data is growing exponentially

Company confidential – distribution prohibited without permission

 Agenda

Live Q&A

Company confidential – distribution prohibited without permission

 Knowledge Center

www.lionbridge.com
Download White Papers
• Building Stronger Brands Around the World: A Guide
to Effective Global Marketing
• Strengthening Global Brands: Key Steps for
Meaningful Communications around the World
• Building a Global Web Strategy: Best Practices for
Developing your International Online Brand
• Machine Translation: Saving Time, Lowering Costs,
and Improving Services

View Webinars

• Localization 2.0
• The Art and Science of Global Navigation
• The Best Global Web Sites (and Why)

Company confidential – distribution prohibited without permission

 Questions?

Nicholas McMahon
VP, Global Technology Solutions
Nicholas.McMahon@lionbridge.com

http://localization2dot0.lionbridge.com

http://blog.lionbridge.com

http://twitter.com/Lionbridge

Lionbridge
www.lionbridge.com

Company confidential – distribution prohibited without permission