iPhone Testing:

This article started out as a Top 5 list but quickly grew as we researched all the tips and techniques we
use at RTL for iPhone application testing. Most of these tips came from tracking down memory-related
problems, which often resulted in defects that were very difficult to capture.

1. AccurateIy report avaiIabIe memory. Many of the non-reproducible bugs you run into when testing
iPhone apps are related to memory problems. t's critical that you know and report available free memory
before launching an application. n all likelihood, the reproducibility of a crashing iPhone app bug is
related to low memory conditions. Consequently, a crashing defect may disappear when there's plenty of
free memory. n a previous article (Using Memory Sweep for iPhone App Testing) we described a tool
that can be used to determine free memory.

2. Provide 'crash reporter' Iogs with your defect reports. Each time an iPhone application crashes,
a.7,8 file is created on your iPhone. You can retrieve this file when you synch your iPhone with
iTunes. Here's a link that describes where those files are stored: iPhone Crash Logs

. Spy on the app from the consoIe. iPhone apps will report application and system level warnings to
the console. You can view these warnings in real-time using Apple's iPhone Configuration Tool. By
knowing what's being reported when interacting with an app can help you refine the steps you need to
reproduce tricky (and memory related) problems.

4. Test under Iow memory conditions. This relates to #1 above. You'll be able to tease additional
crashing bugs if you force free memory to a very low level, e.g. < 2MB, before proceeding with your tests.
One way to do this is to open several Safari windows before you start your testing.

5. Screenshots, screenshots, screenshots. Nothing makes a U bug stand out for a developer than
when you send screenshots. And with the iPhone's built-in screen capture, there's no excuse not to do
this.

6. Provide usefuI defect characterization information. Developers always like to have help in their
debugging process, and useful defect characterization helps them narrow down the root cause of a bug.
f a crash happens under low memory conditions (see #1 and #4 above), then try it under conditions
where there's lots of memory available, e.g. >40MB. f a problem occurs under iPhone OS 2.2.x, then try
it under 3.x.

7. Create connectivity probIems. f you're testing an iPhone app that depends on internet connectivity,
then test for degraded or unavailable connectivity. t's easy to make connectivity unavailable by simply
turning on Airplane Mode. To degrade connectivity, especially on Edge or 3G, employ some sort of
metallic "shield" on top of your iPhone.

8. Boundary test data input. Wherever an iPhone app asks for text input, you have an opportunity to
find a bug. My favorite technique for this is to copy a huge amount of text, then paste it into each text
field. You'd be surprised at how this trips up some apps.

Additionally, we've been finding that application errors are generating when entering the following
characters into text fields: !@#$%^&*()_. (Note: Holding down letters (A, E, , O, etc) or symbols ($, !, &,
etc) on the onscreen keyboard generates a keyboard popup that includes localized and 2-byte
characters. These should also be entered into text fields.)

. Gather up UDIDs (unique device identifiers) earIy. This is a simple logistics task but seems to be
one that becomes critical as the first build approaches. And it's a hassle for the dev team to add new
UDDs and create new provisioning files as each new person wants to install an application during
development. Get the UDDs of all know devices that will be used during testing and set a cut-off date for
the addition of any new devices. Check out Find UDD with a single click. You can also connect all your
iPhones and iPods touches to your computer and use the iPhone Configuration Tool to collect UDDs.

10. EmpIoy background appIications. But the iPhone can only have one application running at a time,
right? That's true for those of us that develop and test applications, but not for Apple. Applications that
continue running in the background on the iPhone are Safari, iPod and Mail. And what about reminders
and push notifications? These "interrupters" can affect the behavior of an application under test.

Also, since iPhones and iPod touches are devices that users buy primarily to use as a phone or a music
player, it's important to test that an app can gracefully handle situations where the user receives a call or
plays music from their music library (iTunes) while the app is running. We've seen issues here where
apps aren't smoothly multi-functioning in these areas.