Introduction to

Mobile Analytics
Andrew Owens
September, 2013
Agenda
Rules of Thumb
The Mobile Ecosystem
Third Party Solutions
The Analytics Lifecycle
Analytics in the Mobile Development
Process
Standard Reports & Dashboards
Statistical Techniques & Tools
Out of Scope Items
Mobile Web Analytics
Covering all mobile analytics tools
The mathematics behind A/B Testing
Rules of Thumb
Business Intelligence is not Analytics
Every aspect of a product is a hypothesis about what
users want
Request feedback on your idea and product often
Focus on short, simple feature releases
Don't go live without analytics.
{"In God we trust; all other must bring data."}
- W. Edwards Deming
The Mobile Ecosystem:
Integrated Apps:
Standalone Apps:
Examples: Virtuoso v. ZipCar
APIs
Devices
Database
The Mobile Ecosystem: iOS &
Android
Devices:
iPad(s)
iPhone(s)
Languages:
Objective-C
Devices:
Gallery
Languages:
Java, XML,
SQLite
The Mobile Ecosystem: Cross
Platform - Titanium & PhoneGap
Devices:
Android, iOS, Windows
Languages:
Javascript, CSS, HTML
The Mobile Ecosystem:
{'There can be significant performance
differences on different technology platforms
and hardware that impact data gathering'}
{'There are limitations to the amount of
campaign/marketing channel data that can be
collected depending on the tool one uses'}
The Mobile Ecosystem: Data
Sources
Advertizing
iAd (iOS)
AdMob (All)
AdWords (All)
In-app purchases
iOS
Android
Link Sharing Channels
LinkShare
CRM Tools
Mandrill
Mail Chimp
Services APIs
Facebook
Twitter
Google
Twilio
Stripe
Databases
Third Party
Analytics Solutions
Third Party Solutions
Our Focus:
List of Mobile Analytics Tools
Events v Pageviews
No Yes
Yes Yes
Pageviews Events
Google Analytics
Key Features:
It's free
Many people already have a background using the tool
Tracks the number and type of application crashes
Campaign measurement parameters are pre-written
Drawbacks:
"The Apple App Store does not currently support campaign measurement
using Google Analytics." (This does not apply to the Google Play Store)
Not as customizable as other analytics tools
There are issues with the existing plugins for cross-platform tools
(Titanium, PhoneGap)
Google Analytics: Events v.
Pageviews
MixPanel
Key Features:
Very customizable
Standardized reports/dashboards are easy to setup and adjust
Drawbacks:
Costs are incurred as collection increases
Campaign measurement parameters must be defined programmatically
Doesn't measure pageviews
The Analytics Lifecycle
So, what do I need to get started?
1. An App project started
2. Choose & set up an analytics solution account
3. Install the analytics solution resources
4. Determine the parameters you're going to collect within
your application
5. Implement the data collection parameters throughout
your app
6. Test to ensure your parameters are collecting data
7. Start your analysis through standard reports &
dashboards
Development: The Mobile
Development Cycle
Requirements & Design Build Test Deploy
Analysis &
Measurem
ent
Product Change Orders
Managing Product Changes
Identify the reason for the change in user
hypothesis with data
Review your end to end functional &
technical requirements and understand how
the change will impact them
Quantify the time and monetary cost of the
change
Standard Reports & Dashboards
MixPanel: Google Analytics:
Data Points
Standard Reports:
Engagement:
Events, User Details, Search Terms, Funnels,
Trends, etc.
Demographic Information:
Country, City, Language
Technology:
Phone Model; Operating System; Network
Funnels
Funnels are extremely helpful in examining key
user goals within the application
Formulas
Formulas allow you to establish Key
Performance Indicators that can be tracked
over time that are not standard reports
Dashboards & Bookmarks
Dashboards & Bookmarks allow for reports to
be saved and even emailed out at a normal
cadence
Google Analytics Campaign
Parameters
*Reminder: Campaign measurement parameters will differ between app ecosystems
Social Media Integration
Facebook:
Build a page
Like, Share buttons
Twitter:
Choose your handle, hashtag
Tweet, follow buttons
Google+:
Build a page
+ buttons
LinkedIn:
Build a page
Share, like buttons
{'Effective social media
integration promotes
branding, viral loops,
CRM, and is essential to
consumer app growth.'}
Others:
Instagram
Flickr
Tumblr
Techniques & Tools: A/B Testing
"A/B testing is a methodology in advertising of using randomized experiments
with two variants, A and B, which are the control and treatment in the controlled
experiment." - Wikipedia
A/B testing is designed to measure the effect of very small changes on user
goals
Why is this different in Mobile than Web?:
Outcomes on websites can be programmatically updated realtime for users
Adjustments to mobile apps need to be resubmitted
A/B Testing Example
https://www.lucidchart.com/documents/view/41ea-f704-515c7c3d-a817-22480a0045e5
Hypothesis:
An increase in the size of the logo on the
sign in page of the application will cause a
significant change in the amount of people
who login to the application using the "Login
with Facebook" button.
Measurement Tool:
MixPanel
Test Requirements:
Users opening the application will be
assigned to either a user profile of Group A
or Group B upon launching the application
for the first time. They will then be randomly
placed into either Group A or B and will
navigate to screen 1.1.A or 1.1.B based on
that assignment.
A/B Testing Example
https://www.lucidchart.com/documents/view/41ea-f704-515c7c3d-a817-22480a0045e5
Signin Screen of App:
Group A:
Mixpanel *mixpanel = [Mixpanel sharedInstance];
[mixpanel track:@"Signin App A"];
Group B:
Mixpanel *mixpanel = [Mixpanel sharedInstance];
[mixpanel track:@"Signin App B"];
Selection of Login w/Facebook:
Group A:
Mixpanel *mixpanel = [Mixpanel sharedInstance];
[mixpanel track:@"FB Click 1.1.A"];
Group B:
Mixpanel *mixpanel = [Mixpanel sharedInstance];
[mixpanel track:@"FB Click 1.1.B"];
A/B Testing Example
https://mixpanel.com/labs/split-test-
calculator
Group A Group B
Signin to App 455 545
Login With Facebook 43 95
Click Through Rate (CTR) 9.45% 17.43%
Let's say we end up measuring this type of
experiment over a month and we have 1000 app
Signins over that period of time.
We know that the CTR is higher in Group B, but is
it high enough for us to be confident in our test?
A/B Testing Example
https://mixpanel.com/labs/split-test-
calculator
How do I take my skills expand my
mobile analytics skillset?
Setup mobile analytics in your app!
Google Analytics Resources
MixPanel Education
Learn about common analytics experiments
Agenda Review
Rules of Thumb
The Mobile Ecosystem
Third Party Solutions
The Analytics Lifecycle
Analytics in the Mobile Development
Process
Standard Reports & Dashboards
Statistical Techniques & Tools
APPENDIX
The App Ecosystem: Platform
Choice
iOS Android
Mobile Web
Examples:
The App Ecosystem: Platform
Choice (Continued)
Examples:
PhoneGap
Titanium
More: http://mashable.com/2012/03/28/mobile-app-development-advice/