1

MALAYSIA USER SATISFACTION

EVALUATION (MUSE) 2016
WRAP-UP SESSION

8TH DECEMBER 2016

SATRIA1, DORSETT PUTRAJAYA
 2

42nd FCC Dialogue:ACCELERATING EFFORTS TO IMPACT

EXCELLENCE IN GOVERNMENT ONLINE SERVICES DELIVERY

“We need to be agile in navigating digital

transformation and always be conscious that
what we will do will benefit the citizens of
Malaysia”

YBhg. Tan Sri Dr. Ali Bin Hamsah

Chief Secretary to The Government of Malaysia
42nd Flagship Coordination Committee Dialogue,
Sama-Sama Hotel KLIA,
22nd November 2016
 3
Recap: Measuring User Satisfaction on GOS
Towards Citizen-centric Services (38th FCC – 19th Nov 2014)
“MDEC to undertake on audit on all of the Government agency websites and
advise YAB Chairman exactly where they are in the websites and propose ways to
enhance and upgrade the websites” 19th ICM on December 7th 2005

SUPPLY/PROVIDER DEMAND/USER
CENTRIC CENTRIC

Provider-Based Evaluation Malaysia User Satisfaction

(ProBE) Evaluation (MUSE)
2014 Onwards 2015 Onwards

Contributed to the improvement MUSE Baseline 2015 focuses

and increase of online services… on two key components:

No. of online From <10% Extent of Usage

public services to 71%

Standardisation From 13% Level of Satisfaction

(3-STAR RATING & ABOVE) to 92%
 4

42nd FCC Dialogue:ACCELERATING EFFORTS TO IMPACT

EXCELLENCE IN GOVERNMENT ONLINE SERVICES DELIVERY

Governments can no longer afford to continue the

conventional practices to imagine the needs of the users
or deriving conclusions from insufficient data that does
not reflect the actual needs, nor procuring systems to
implement an already defined public policy.

Unfortunately, governments infrequently consider the

potential to offer a similar level of services provided by
the private sectors such as online shopping and online
banking.

STEPHEN FORESHEW-CAIN
Former Executive Director – Digital Government Service (GDS) UK
42nd Flagship Coordination Committee Dialogue,
Sama-Sama Hotel KLIA,
22nd November 2016
 5

MUSE 2016/2017 Implementation

6
 41ST FCC DECISION, APRIL 2016
Decision: Scope for 13 pilot agencies & 1 myGov portal to be
7

expanded & to include agencies under MOF, MOHR, NRE & KPWKM as
part of survey coverage.

MOF
10
1. Jab. Akauntan Negara
2. Jab. Peniliaian &
24
(23 Agencies
Perkhidmatan Harta
and 1 portal)
KPWKM
3. Jabatan Kebajikan
Masyarakat (JKM)
4. Lembaga Penduduk
dan Pembangunan
Keluarga Negara
(LPPKN)
MOHR
5. JobsMalaysia
6. Jab. pembangunan
Kemahiran
7. JKKP
NRE
8. Jabatan Pengairan &
Saliran Malaysia (JPS)
9. Jabatan Perhilitan
10.Jab. Ukur & Pemetaan
Msia (JUPEM)
MUSE BASELINE 2016 FRAMEWORK

Sentiment
Analysis

• Gather and
analyse relevant
social media
channels to
understand and
elicit insights
 MUSE 2016 Snapshot

15th
JUNE
24 12K 26 491

SURVEY AGENCIES RESPONDENTS AGENCY ONLINE SERVICES

COMMENCEMENT INVOLVED ACHIEVED ENGAGEMENT ASSESSED
100% SESSIONS (subject to change)
(~400 PARTICIPANTS)

Note: Comparison with MUSE 2015:

• Commencement Date: 1 Aug 2015
• Involved 14 agencies
• Total of 7,455 respondents (GOS)
• 327 online services assessed
 10

MUSE 2016 PRELIMINARY RESULT

 11

SURVEY DEMOGRAPHIC
Ethnic Group Education Monthly Household
Gender
Income

55% 84% 26% >RM3k 31%

72% 25.5%
51.5% 35%

9%
45% 18% 74% RM3-
RM5k
34%
30%
48.5% 74.5%
7%
5%
2016 result
RM3-
RM5k
35%
35%
2015 result

Age Occupation Type

26% 64% 10% Government / Private / Others

Statutory Bodies Self Employed
37.5% 55.5% 6.5% 56% 35% 6%
46% 48% 5%

15-29 30-49 50-69

 MUSE 2016 RESULTS AND FINDINGS
GOVERNMENT ONLINE SERVICE USER SATISFACTION INDEX

77% (2015: 75%) Extremely Satisfied / Satisfied

T2B %

ACCESSIBILITY 1%3% 16% 53% 27% 80% (2015: 77%)

PRIVACY/
SECURITY 1%
2% 20% 54% 24% 78% (2015: 77%)

WEB DESIGN 1%4% 18% 55% 22% 77% (2015: 77%)

EASE OF USE 1%4% 18% 53% 24% 77% (2015: 77%)

INFORMATION 1%3% 18% 53% 24% 77% (2015: 78%)

PERSONALISA 75% (2015: 62%)
TION 1%3% 21% 55% 20%

RELIABILITY 1%4% 20% 53% 22% 75% (2015: 75%)

RESPONSIVE 1%5% 21% 52% 21% 73% (2015: 72%)

1 -EXTREMELY DISSATISFIED 2 - DISATISFIED 3 - NEUTRAL 4 - SATISFIED 5 - EXTREMELY SATISFIED

12
Base: Amongst All Respondents n= 12,293 (respondents of 24 participating government agencies)
 MUSE 2016 vs. 2015 RESULTS
GOVERNMENT ONLINE SERVICE
USER SATISFACTION INDEX
ACCESSIBILITY
77% (2015: 75%) 80
70
EASE OF USE 60 WEB DESIGN
Most satisfied 50
40
2016 2015 30
1. Accessibility 1. Information 20
2. Security 2. Ease of Use 10
3. Web design 3. Accessibility INFORMATION 0 RELIABILITY

Least satisfied
2016 2015
PERSONALISATION RESPONSIVE
1. Responsive 1. Personalisation
2. Reliability 2. Responsive
3. Personalisation 3. Reliability
PRIVACY/SECURITY

OVERALL 2016 OVERALL 2015

13
 MUSE 2016 RESULTS AND FINDINGS:
AGENCIES PERFORMANCE (SATISFACTION)

2016 2015 2016 2015

Front Line Agencies UN- Focused Agencies
Satisfaction Satisfaction Satisfaction Satisfaction

KWSP 87% 77% MOE 65% 75%

LHDN NRE 88% 75%

83% 77%

JPJ 74% 75% MOHR 97% 74%

JIM 84% 74% MOF 76% 73%

JPN 86% 74% KPWKM 79% 73%

Perkeso 78% 73% MOH 74% 72%

JKDM 74% 66% MyGov 70% 69%

For agencies involved in both years, 6 obtained > 80% satisfaction level in 2016
 MUSE 2016 RESULTS AND FINDINGS:
AGENCIES PERFORMANCE (SATISFACTION)

2016 2016
MUSE 2016 Agencies MUSE 2016 Agencies
Satisfaction Satisfaction

LPPKN 84% JPS 85%

JANM 86% JUPEM 79%

JPK 60% JKKP 71%

.Jobs
72% PERHILITAN 30%
Malaysia

JKM 75% JPPH 87%

From newly included agencies, lowest satisfaction level is 30% &

4 agencies exceeded 80%
15
 USER PERCEIVED EXPECTATION
Respondents participating in this survey have fairly high expectations of government online service.
In overall, however, the actual online experience (57%) was short of expectation.

PERCEIVED EXPECTATION OF ACTUAL PERFORMANCE OF

GOVERNMENT ONLINE SERVICE GOVERNMENT ONLINE SERVICE

63%
50% 57%
47%

37%

32%

13% 10%
5%
4% 1%
1%
1- MUCH 2 - LESS THAN 3 - AS PER 4 - MORE 5 - GREATLY
1 - VERY 2 - LOW 3 - NEUTRAL 4 - HIGH 5 - VERY
LESS EXPECTATION THAN EXCEEDED
LOW HIGH

Q3.1. Please rate your expectation level of GOVERNMENT ONLINE SERVICE in general? You may select
any number from 5 to 1, where 5 means “Very High Expectation” to 1 means “Very Low Q3.2. Now based on your experience with GOVERNMENT ONLINE SERVICE, how well did the experience
meet your expectation?
Expectation”
17
Base: Amongst All Respondents n= 12,293 (respondents of 24 participating government agencies)
 DRIVERS OF SATISFACTION
Based on the regression analysis, it is identified that “Ease of Use” is the key dimension influencing
the satisfaction of online service experience.
Share of importance
(out of 100%)

EASE OF USE 29% Rank 1

INFORMATION 16% Rank 2

Importance ranking
ACCESSIBILITY 12% Rank 3 (measures the impact of
each dimension on
overall satisfaction)
RESPONSIVE 11% Rank 4
Regression provides
PERSONALISATION 11% Rank 5 standardized coefficient
which are indexed to
100%
RELIABILITY 8% Rank 6
The indexed are then
WEB DESIGN 7% Rank 7 used to calculate the
share of importance of
Rank 8 each dimension.
PRIVACY/SECURITY 6%

Base: Amongst All Respondents n= 12,293 (respondents of 24 participating government agencies)

 ROOT CAUSE ANALYSIS
SATISFACTION AREAS OF IMPROVEMENTS
SCORE

Access to the website whenever needed 50%

ACCESSIBILITY 80% Access to the website using different devices 45%
Access to the website using various browsers 37%

Regular update of information 64%

INFORMATION 78% Easy to understand 52%
Information accuracy 50%

Clearly displayed information and services 53%

WEB DESIGN 77% Consistent layout throughout the website 43%
Properly organised web page structure 42%

Continuous improvement on the level of online security 57%

PRIVACY/SECURITY 77% The level of security of online transactions 44%
The protection of personal information 42%

Easy to search online information or services 52%

Dimensions are ranked in EASE OF USE 77% Website functions enable to complete my transaction with
51%
accordance to ease
satisfaction score Easy to navigate from one page to another page 50%
Completing transactions without any disruption 52%
USER SATISFACTION RELIABILITY 75% Receives consistent service every time 48%
INDEX Completing transactions in a reasonable time 39%

77% Automatic suggestions to related links / services 48%

PERSONALISATION 75% Ability to assist in completing transaction 48%
Customisation as per personal preference 44%

Receives prompt feedback for online enquiries 56%

RESPONSIVE 73% Receives alert notification when an error occurs 50%
Receive sufficient guides to complete online transaction 46%

Base: Amongst All Respondents n= 12,293 (respondents of 24 participating government agencies)

 AREAS OF PRIORITISATION
“Information” and “Ease of Use” are the two key areas requires Prioritisation in improvement efforts.
Specifically is focusing on “Easy to search online information or services” and ‘providing regular updates
on information’.

High

Areas to
Drivers of Satisfaction / Maintain Strength to build on
Importance
Importance Performance
Rank
% %
Accessibility

ACCESSIBILITY 3 12 80

Satisfaction/Performance
Privacy/Security
WEB DESIGN 7 7 78 Web Design

PRIVACY/SECURITY 8 6 78 Ease of Use

Information
EASE OF USE 1 29 77
Reliability
INFORMATION 2 16 77
Personalisation
PERSONALISATION 5 11 75
Responsive
RELIABILITY 6 8 75
Key Priorities for
Secondary Action
Priorities
RESPONSIVE 4 11 73
Low High
Drivers of Importance

Base: Amongst All Respondents n= 12,293 (respondents of 24 participating government agencies)

 GOVERNMENT vs. PRIVATE ONLINE SERVICES

GOVERNMENT ONLINE SERVICE VS. PRIVATE ONLINE SERVICE WORSE SAME BETTER

OVERALL PERFORMANCE 6 50 43

WEB DESIGN 11 47 41

PRIVACY/SECURITY 7 51 41

RELIABILITY 9 51 41

INFORMATION 7 48 45

ACCESSIBILITY 9 48 44

RESPONSIVE 11 50 39

PERSONALISATION 8 52 40

EASE OF USE 7 50 42

• 43% claimed the performance to be ‘better’ and 50% claimed to be the same.

• ‘Information’ and ‘Accessibility’ are two parameters which are rated fairly high
compared to others when comparing with private online service.

Base: Amongst All Respondents n= 12,293 (respondents of 24 participating government agencies)

GOS e-Rating PRELIMINARY RESULT
 GOS E-RATING (INVOLVING 91 ONLINE SERVICES)
Satisfaction Net Promoter
Rank Government Agencies Respondents
Score (%) Score (NPS)

OVERALL 61,573 77 5
2016 1 PERHILITAN 104 93 18
2015
77% 2 KWSP 5251 92 37
81% Satisfaction
Satisfaction 3 JKM 122 92 43
level
level 4 JPK 82 91 -98
5 MOE 14420 91 26
6 JIM 1480 87 45
7 JobsMalaysia 3936 78 11
8 JUPEM 960 78 -7
GOS e-Rating 9 JPN 784 77 23
Least Satisfied: 10 JANM 11056 77 2
2015 2016 11 PERKESO 298 70 -7
• Accessibility • Search 12 LHDN 9781 70 -13
• loading time • Responsive 13 MOH 1000 67 -15
• effortless service • Security 14 MOF 3838 63 -24
15 JPS 65 62 -11
MUSE Survey 2016: 16 MyGov 87 57 -38
Least satisfied Most Satisfied 17 JPJ 6287 57 -20
• Responsive • Accessibility 18 JKKP 1934 56 -21

• Reliability • Privacy 19 JKDM 73 36 -48

• Personalisation • Web Design

Note: Public Sector NPS Global Benchmark 22
23
ONLINE STATISTIC PRIMILINARY RESULT
 MUSE 2016: TOP 20 ONLINE SERVICES
Agency Services Total Transactions
JANM e-Penyata Gaji & Laporan 20,234,144
JANM e-Maklum 15,769,544
Total Online LHDN E-BR1M 15,685,598
Services: MOE Penjanaan Laporan 10,478,647
2016 – 491 MOE Kemaskini Maklumat Guru 9,525,743
KWSP Login ke I-Akaun Majikan 8,405,510
2015 – 327 JIM Semakan Status Perjalanan Ke Luar Negara (SSPI) 8,393,222
KWSP Semak Penyata i-Akaun (Ahli) 8,051,179
KWSP Login ke I-Akaun Ahli 7,837,271
eDeclare (Pengikraran Kastam) - National Single
MOF 7,332,850
Window – MyTradelink
Total volume: JANM e-Pinjaman Komputer 5,495,104
2016 - 190Mil KWSP e-Caruman/Caruman-Borang A (Majikan) 5,220,742
2015 – 69Mil KWSP Semak Maklumat Penamaan di I-Akaun Ahli 4,857,540
MOF eManifest (Penghantaran Maklumat Perkapalan) 4,555,367
LHDN E-Filing: Individual/ company/ others income tax filing 4,351,887
JKDM e-Declare K2 : Kastam 2 3,724,197
Data is collected
from Nov 2015 – Oct LHDN MyTax - Income Tax Notification/ Status 2,762,237
2016 LHDN STAMPS 2,505,329
JPS Infobanjir : Flood Monitoring System Realtime rainfall
and waterlevel data. Useful for indicators of potential 2,417,566
flooding or landslides.(page loads)
JKDM e-Declare K1 : Kastam 1
2,398,720
 26

Moving Forward

26
 MOVING FORWARD

1. Expansion of MUSE Agencies for MUSE 2017 implementation

Expansion of MUSE Agencies (tentative)

14 24 70
2015 2016 2017

2. Collaboration with Indeks SSR

3. Promotion and awareness of GOS

27
28

Thank You

28