SCE V2 Product Background

Mission: Provide on-premise unified management solution specifically designed for IT Generalists in midsize businesses to efficiently manage and secure their IT environments.
Strategic Objectives Deliver unified IT management offering for midsize businesses Easy-to-use user experience from install to administration Offer complete features required by the IT Generalist On-premise management solution for customers IT Generalist

Role is a mile wide and an inch deep.

Medium IT organizations consists of two-to-four full-time administrators

Drive adoption of SC management solutions within the midmarket

Provide superior solutions through integrating existing technologies Utilize proven enterprise innovations Design integration for simplicity Role Span of control Day-to-day management and troubleshooting of network, servers, OS, and applications Provides high level of end-user support All company servers, desktops and network MCSE training and self-taught

IT Experience

UX Challenges and Plan of Record

Tight timeline for UX: Getting management and product teams to

understand that you can win market share with UX Product team only wanted to work on new features: Use benchmark data to drive improvements in the new version for existing scenarios First UX engagement for the product team: Realizing that UX is evolutionary, not revolutionary Focus first on smaller, high-visibility wins Heavily involve PP and PM in the process Many consistency problems across the product: Dedicate a group to focusing on 2-3 small areas of the overall vision Must show management that the whole is greater than the sum of the parts (reuse patterns and design). Do end to end consistency pass
3

UX Goals & Success Criteria

GOALS Deliver high quality user experience for new investments in SCEv2 Deliver high quality experiences for v1 outstanding issues Drive and adhere to Guidelines and Framework project SUCCESS CRITERIA New v2 designs go through the UX process All P1 usability findings incorporated Use benchmarking data to drive v2 improvements

Product Milestones and Timeline

UX Milestones and Timeline

SCE UX Process and Deliverables

SCE Research Studies and Deliverables

SCE V1 BENCHMARK STUDY DRILL DOWN

Benchmark study: Goals

To record user success/failure/satisfaction on core

scenarios with V1 product To identify V1 UI with usability problems in the context of core scenarios. To use as a yardstick to measure improvement and direct simplicity efforts in V2

Benchmark Study Process

Key scenarios that needed to be tracked in SCE V1 were defined with the product team. Success metrics based on the tasks and goals were also defined. 24 IT Generalists were recruited for this study. Activities were timed Participants were then asked to perform a set of tasks on using SCE. The experimenter captured whether the participant was successful as well as the time taken. Overall satisfaction and perceived ease of use was also collected using the standard post-task questionnaire. Participants were asked to fill out a product survey at the end to gather data on the four subjective metrics of satisfaction for the product.

Benchmark study: Methodology

Task Success: What percentage of participants was able to successfully

complete a given task/scenario?

Experience Scores: Subjective metrics of satisfaction post task and post

study: What percentage of participants either agreed or strongly agreed with statements covering the following four aspects of satisfaction?
Satisfaction with overall experience (required for all P1 tasks) Perceived Ease of Use (required for all P1 tasks) Perceived Functionality (required for each distinctly different task) Perceived Visual Appeal (required for each distinctly different task)

Success criteria for each of these measures was defined as follows: Green >= 85 %, Yellow >=70% , Red <70%

Benchmark study: UX Measures

Benchmark study: Post task Questionnaire

Required for all tasks I am satisfied with the overall experience of completing this task. I found the program easy to use in this task. Strongly Disagree Strongly Disagree Disagree Neither agree nor disagree Neither agree nor disagree Agree Strongly Agree Strongly Agree Disagree Agree

Required for all distinctly different tasks (either in terms of functionality or UI):
I think the functionality provided by the program in this task would be useful for my job. I found the program visually appealing in this task. Strongly Disagree Strongly Disagree Disagree Neither agree nor disagree Neither agree nor disagree Agree Strongly Agree Strongly Agree

Disagree

Agree

Benchmark study: Post task Questionnaire

Required for all tasks I am satisfied with the overall experience of completing this task. I found the program easy to use in this task. Strongly Disagree Strongly Disagree Disagree Neither agree nor disagree Neither agree nor disagree Agree Strongly Agree Strongly Agree Disagree Agree

Required for all distinctly different tasks (either in terms of functionality or UI):
I think the functionality provided by the program in this task would be useful for my job. I found the program visually appealing in this task. Strongly Disagree Strongly Disagree Disagree Neither agree nor disagree Neither agree nor disagree Agree Strongly Agree Strongly Agree

Disagree

Agree

Benchmark study: Subjects and Scenarios

24 Core Morg IT Professionals mapping to the Chris Green persona Participants had no previous experience with System Center

Scenario - Recruiting criteria Initial configuration: Have set up a new server in the past 2 years Software Deployment: Have deployed new software in the past 2 years Update Deployment: Deploy or install updates on machines at least monthly Server Migration: Have migrated a server in the past 2 years Troubleshooting : Have conducted server troubleshooting in the past 2 years

Benchmark Results
Scenario Success Experience

Product score
Configure SCE Software Deployment Alerts and troubleshooting Asset viability and inventory Update deployment

85%
86% 99% 78% 99% 61%

80%
83% 97% 71% 93% 54%

Integration with the product team

PMs, Devs were encouraged to watch lab sessions Links and nuggets of information such as user

quotes were sent almost on a daily basis High level findings and key videos were presented two weeks after the study Benchmark score was provided for early feature prioritization meetings

Benchmark Impact
Impact on feature and resource prioritization for

SCE V2 Survey questions incorporated in the lab studies to measure success metrics Regular benchmarks are now part of the release cycle Benchmark guidelines and process were documented based on the impact

Detailed Results: Task 1.0: Initial Configuration

You have just purchased and installed System Center Essentials and would like to start running it in your organization. Complete all required System Center Essentials configuration to allow you to start managing your network. Task shouldnt take longer than ten minutes in order to be successful
1.0 Set up and Discovery 1.1 Configure SCE

Success Rate
86%

Experience Rate
83%

Time (min)
6.4

Detailed Results: Task 2.0: Update Deployment

It is the end of the month and you would like to make sure that your machines are patched and updated.
Determine which new Security or Critical updates are available that have not yet been deployed to machines in your network. Deploy security update KB929969 to all client machines with a deadline for tomorrow at 1am. Check that each of the following updates have been approved and installed to all desktops: Security update for Windows Vista KB930178 Security update for Windows Vista KB925902

You would like some of your computers to get updates without the need to manually approve them. Configure Systems Center Essentials to manage updates in the following way:
Download updates for Windows XP and Windows Server 2003 Standard Download only Critical Updates by default For desktops, updates should be downloaded and installed automatically every day at 2:00 AM. Your desktop computers get all critical and security updates automatically. Updates for servers are manually approved

You would like to periodically see all Critical updates, Security updates and Service Packs for Windows Server 2003 that your computers need but have not yet been approved. Create a permanent view to view only these updates and service packs.

Detailed Results: Task 2.0: Update Deployment

Task Determine available Updates Deploy Update (approve for deployment) Determine if Update install was successful Configure Automatic Updates Create Update view Success Rae 63%
55%

Experience Score 61%

43%

Time (min) 4.2

1.1

67%

75%

1.2

39%

25%

10.8

79% 71%

68% 54%

3.1 20.4

Benchmark study: Sample Post study Questionnaire

Questions around overall experience
I am satisfied with the overall experience of using the program. Strongly Disagree Disagree Neither agree nor disagree Neither agree nor disagree Neither agree nor disagree Neither agree nor disagree Neither agree nor disagree Neither agree nor disagree Agree Strongly Agree

I am satisfied with the amount of time it took to complete the tasks. The program requires too many steps to accomplish some tasks. I found the program easy to use.

Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree

Disagree

Agree

Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree

Disagree

Agree

Disagree

Agree

I found the program unnecessarily complex.

Disagree

Agree

The error messages were confusing.

Disagree

Agree