A NDROID A PPLICATIONS T EST

Test Plan
Android testing has several approaches : - Test Android implementations : its the object of CTS - Test mobile application : its the object of this document To complete these tests, the user can use tools : - CTS has automated the test plan - This document proposes a test plan manually ran but during the event different solutions to partially automate these tests will be tested

258067

MOSQUITO

Android Test Plan

Sommaire
1. INTRODUCTION 2. P RE-TESTS 3. USER INTERFACE 5 6 7

3.1. Delivery .................................................................................................................................. 7 3.2. Running .................................................................................................................................. 7 3.3. Splash screen .......................................................................................................................... 7 3.4. Screens................................................................................................................................... 7 3.5. Sounds, colours & vibra .......................................................................................................... 9 3.6. Registration system ................................................................................................................ 9 3.7. Lisibility .................................................................................................................................. 9

4.

LNTERACTING WITH THE OS

11

4.1. Resume ................................................................................................................................ 11 4.2. Interferences ........................................................................................................................ 11 4.3. SMS ...................................................................................................................................... 12 4.4. System ................................................................................................................................. 13

5. MULTIMEDIA

15

5.1. Protocol ............................................................................................................................... 15

6. NETWORK

18

6.1. Protocol ............................................................................................................................... 18 6.2. Communication .................................................................................................................... 18 6.3. Security ................................................................................................................................ 18

7. S TRESS TEST

19

7.1. Memory ............................................................................................................................... 19 7.2. Data ..................................................................................................................................... 19 7.3. Interruption .......................................................................................................................... 20 7.4. User interactions .................................................................................................................. 20

8. D OCUMENTATION

21

8.1. Documentation ..................................................................................................................... 21 8.2. Minimum requirements ........................................................................................................ 21 8.3. Interaction ........................................................................................................................... 21 8.3.1. Advice ..................................................................................................................................................21 MOSQUITO Page 2/23

258067

MOSQUITO

Android Test Plan

9. INTERNATIONAL APPLICATIONS

23

9.1. Localization .......................................................................................................................... 23 9.2. Translation ........................................................................................................................... 23

MOSQUITO Page 3/23

258067

MOSQUITO

Android Test Plan

MOSQUITO Page 4/23

258067

MOSQUITO

Android Test Plan

1.

Introduction

CTS is designed to test Android implementations. This test plan is designed to test Android mobile applications. The test cases are intended to be ran manually. But some parts can be automated with specific tools.

MOSQUITO Page 5/23

258067

MOSQUITO

Android Test Plan

2.

Pre-Tests

This section provides general tests about launching the application. 2.1. Is the application launched successfully ? 2.2. Once launched, does the application gain control ? 2.3. Does the installation of your application not interfere with the pre-installed applications ? 2.4. Can you come back to the application main menu screen in less than 3 seconds ? Yes Yes Yes Yes No No No No

MOSQUITO Page 6/23

258067

MOSQUITO

Android Test Plan

3.

User Interface

This section provides tests about delivery, running the application, display, using and lisibility of the application.

3.1. Delivery
3.1.1. Is the apk file provided ? 3.1.2. For the client server solutions, is the server software available ? Yes Yes No No

3.2. Running
3.2.1. Does the application provide an exit functionnality ? 3.2.2. If the application is not launched within 6 (six) seconds, is a warning message or a progress bar displayed ? 3.2.3. If the application is resumed from a suspended state, does it return to this state ? 3.2.4. When at the application main menu, is the application suspended ? Yes Yes No No

Yes Yes

No No

3.3. Splash screen

3.3.1. If there is a splash screen, is it displayed only during the application launch ? 3.3.2. Is the splash screen displayed for long enough to read the text ? 3.3.3. Is the splash screen displayed less than three (3) seconds ? Yes Yes Yes No No No

3.4. Screens
3.4.1. Does the first screen contain the application name or title in a title bar ? Yes No No No

3.4.2. Are the screens properly refreshed after Yes dismissing any another screen ? 3.4.3. In the text fields, is the insertion point Yes MOSQUITO Page 7/23

258067

MOSQUITO

Android Test Plan

automatically placed at the end of the field ? 3.4.4. Is the delay to display the next screen Yes enough short (e.g. less than 10 seconds) ? 3.4.5. If your application display icons, are they clear, visible and appropriate ? 3.4.6. Is the number of screen to browse minimized ? 3.4.7. Are each screen displayed for time enough to read information ? 3.4.8. Are the screens easily readable ? 3.4.9. Are the pre filled forms used in order to minimize the amount of writing ? 3.4.10. Is the virtual keyboard displayed in all the text fields of the screen when needed ? 3.4.11. Is each screen correctly displayed in each mode (landscape, portrait, external screen, ...) display mode ? 3.4.12. Is the application display adapted to the screen size ? 3.4.13. During long operation, is an animated visual indication displayed to inform the user ? 3.4.14. If the application play sounds, does the application propose an option to mute sound ? 3.4.15. If the application play sounds, does the application propose an option to mute sound ? 3.4.16. Does the start up screen display the title of the game and allow to access to the functions of the game ? 3.4.17. If the game contains user setting, does the application provide a form to change settings ? Yes Yes Yes Yes Yes Yes Yes

No No No No No No No No

Yes Yes Yes Yes Yes

No No No No No

Yes

No No

3.4.18. Does the High scores screen display a list of Yes scores and the names of the players in decreasing score order ?? 3.4.19. Does the Help screen give the purpose of the Yes game and instructions about the keys to use ? 3.4.20. In the play screen, does the back key allow to back to the start up screen ? 3.4.21. When in paused state, is the "Continue" option the first of this list ? 3.4.22. In the play screen, when the game is over, does the screen allow to save the score, display the total score and notify the game over ? 3.4.23. In the optional load screen, does the Yes Yes Yes

No No No No

Yes

No

MOSQUITO Page 8/23

258067

MOSQUITO

Android Test Plan

application list the saved games ?

3.5. Sounds, colours & vibra

3.5.1. Have sounds a specific function ? 3.5.2. Does the application use sounds without abuse ? 3.5.3. Are colours harmonious ? 3.5.4. If sound is used, is it possible to disable it ? 3.5.5. If vibrator is used, is it possible to disable it ? Yes Yes Yes Yes Yes No No No No No

3.6. Registration system

3.6.1. If the application use an expiration and/or registration system, is the notification displayed once after the installation of the application ? 3.6.2. Once the application has expired, does the registration message disappear ? 3.6.3. If the application has expired, does the expiration message give the option to the user to enter an activation code ? 3.6.4. If the application display the number of remaining sessions or days, is it displayed in "About" dialog or in a specific menu option ? Yes No

Yes Yes

No No

Yes

No

3.7. Lisibility
3.7.1. Are images displayed and refreshed properly ? 3.7.2. Are the texts entirety (not truncated) ? 3.7.3. Is there grammar mistakes ? 3.7.4. Does the text can cause misunderstanding ? 3.7.5. Is the font of the text readable ? 3.7.6. Do selection, scrolling and text highlighting are clear in order to avoid visually confusing ? 3.7.7. Have elements an appropriate contrast for readability ? 3.7.8. Is the whole screen used ? Yes Yes Yes Yes Yes Yes Yes Yes MOSQUITO Page 9/23 No No No No No No No No

258067

MOSQUITO

Android Test Plan

3.7.9. Does each screen indicate the action to perform into the screen (eg. "Load") ? 3.7.10. Does the first letter of the first word of a text always start with a capital letter ?

Yes Yes

No No

MOSQUITO Page 10/23

258067

MOSQUITO

Android Test Plan

4.

lnteracting with the OS

This section tests the resume fonctionnality and the interaction with the handset.

4.1. Resume
4.1.1. Upon receiving an incoming event (e.g. a phone call or a SMS message), does the application loose control ? 4.1.2. If the user has entered data and if the application is going to delete the data without to save, does the application warn the user before ? 4.1.3. If the user has entered data and if an incoming phone call or SMS message arrives, does the application save data if the user quits the application to answer to the event ? 4.1.4. Is the application able to be suspended, then resumed in the same screen and state ? Yes No

Yes

No

Yes

No

Yes

No

4.2. Interferences
4.2.1. If the application update preferences, does it warn the user ? the user Yes Yes No No

4.2.2. Does the application not write to the handset's address book without the authorization of the user ? 4.2.3. Does the application not write to the handset's calendar without the authorization of the user ? 4.2.4. Does the application not change user setting in other applications ? 4.2.5. If there is an incoming call and if the application try to send data on the network, does an error message is displayed ? 4.2.6. If the application reacts at specific events (like focus change, button click, etc ...), is the behaviour correct ? 4.2.7. If the application supports text speech, is the text speech correctly synthesized in the language of the application ? 4.2.8. If the application supports speech recognition, can the application recognize the voice in the language of the application ?

Yes

No

Yes Yes

No No

Yes

No

Yes

No

Yes

No

MOSQUITO Page 11/23

258067

MOSQUITO

Android Test Plan

4.2.9. If the application uses vibrator, is the vibrator correctly activated ? 4.2.10. If the application monitor files, is the access to the files detected by the application ? 4.2.11. If the application uses sensors, is the application correctly driven by these sensors ? 4.2.12. If the application supports gestures, is the gesture correctly read ?

Yes Yes Yes Yes

No No No No No

4.2.13. If the application supports gestures and Yes creating gestures, is the gesture correctly reloaded and read after saving ?

4.3. SMS
4.3.1. If the application initiates a SIP audio call, does the call succeed ? 4.3.2. If the application initiates a SIP audio call, once the call ended, is the link released ? 4.3.3. If the application receives SIP audio calls, does the call is correctly received ? 4.3.4. If the application receives SIP audio calls, once the call ended, is the link released ? 4.3.5. If the application uses SIP audio calls and DTMF tones, do the DTMF tones are recognized ? 4.3.6. If the application uses WIFI network, do the operations done by the application are correctly done (scan, initiate and terminate connections) ? 4.3.7. If the application communicates with a webserver, if the network coverage fails, does the application automatically reinitiates the communication once the coverage is reestablished ? 4.3.8. If the application communicates with a webserver, if the network coverage fails, can the user display the main menu of the mobile phone ? 4.3.9. If the application can send a SMS, is the SMS correctly sent ? 4.3.10. If the application can send a SMS, is the SMS correctly sent three times after ? 4.3.11. If the application can receive a SMS, is the SMS correctly received ? 4.3.12. If the application can receive a SMS, is the SMS correctly received thee times after ? Yes Yes Yes Yes Yes Yes No No No No No No

Yes

No

Yes

No

Yes Yes Yes Yes

No No No No No

4.3.13. If the application uses NFC and supports Yes real NFC tags, are data correctly read from a NFC MOSQUITO Page 12/23

258067

MOSQUITO

Android Test Plan

tag ? 4.3.14. If the application uses NFC and uses Android devices as NFC tags, are data correctly read from the other Android device simulating a NFC tag ? 4.3.15. If the application uses NFC tags, can the application read three times after a NFC tag ? 4.3.16. If the application uses autostart application on receiving a SMS, can the application start ? 4.3.17. If the application uses autostart application on receiving a SMS, can the application start three times after exit ?

Yes

No

Yes Yes Yes

No No No

4.4. System
4.4.1. If the application supports backup/restore feature, if the user wipes data, reinstall the application and restore data, the previous data should be available in the application. 4.4.2. If the application is a widget, can it be installed in the home screen application ? 4.4.3. If geolocation is supported, is the accuracy sufficient ? 4.4.4. If geolocation is supported, does it save battery and data exchange as soon as possible ? 4.4.5. If geolocation is supported and if updates are supported, do the location updates work ? 4.4.6. If the application shares content with another application, can this another application access to these data ? 4.4.7. If the application uses web browsing, can it launch the web browser ? 4.4.8. If the application read contacts, can it read a phone number ? 4.4.9. If the application read contacts, can it read a contact name ? 4.4.10. If the application supports calendar and read events, can it read an event calendar ? 4.4.11. If the application supports calendar and create events, can it create an event calendar ? 4.4.12. If the application read files, can it read a file ? Yes No

Yes Yes Yes Yes Yes

No No No No No

Yes Yes Yes Yes Yes Yes

No No No No No No

MOSQUITO Page 13/23

258067

MOSQUITO

Android Test Plan

4.4.13. If the application write files, can it write a file ? 4.4.14. If the application delete files, can it delete a file ? 4.4.15. If an external application is called (as Google Map), is it displayed in less than 5 seconds ? 4.4.16. When an event happens and the application exists, does the application exit without disturb the mobile phone ? 4.4.17. When an event happens and the application is suspended, does the application run without disturb the mobile phone ? 4.4.18. When the Application is suspended then resumed, if a memory card is inserted, does the Application work correctly ? 4.4.19. When a memory is inserted, does the application work correctly ? 4.4.20. When a memory is removed, does the application work correctly ? 4.4.21. When the application is launched, does it correctly run if the device is closed ?

Yes Yes Yes

No No No

Yes

No

Yes

No

Yes

No

Yes Yes Yes

No No No

MOSQUITO Page 14/23

258067

MOSQUITO

Android Test Plan

5.

Multimedia

This section is applicable for applications that support multimedia.

5.1. Protocol
5.1.1. If Face Detector is used, is the face of one person correctly identified ? 5.1.2. If the application supports external media files and DRM system, can it use a DRMez media ? 5.1.3. If the application support capture image, can it capture one image ? 5.1.4. If the application support capture image, can it capture one image two times after ? 5.1.5. If the application support record video, can it record one video ? 5.1.6. If the application support record video, can it record one video two times after ? 5.1.7. If the application support record sound, can it record one sound ? 5.1.8. If the application support record sound, can it record one sound two times after ? Yes Yes Yes Yes Yes Yes Yes Yes No No No No No No No No No

5.1.9. If the application allows to control volume, Yes can the user set the volume at the lowest level volume ? 5.1.10. If the application allows to control volume, Yes can the user set the volume at the maximum level volume ? 5.1.11. If the application allows to play DTMF Yes tones, are they accepted for the planed usage ? 5.1.12. Can the application display all the supported formats images from the local storage ? 5.1.13. Can the application play all the supported formats videos from the local storage ? 5.1.14. Can the application play all the supported formats sounds from the local storage ? 5.1.15. Can the application display all the Yes Yes Yes Yes

No

No No No No No

MOSQUITO Page 15/23

258067

MOSQUITO

Android Test Plan

supported formats images from the web ? 5.1.16. Can the application play all the supported formats videos from the web ? 5.1.17. Can the application play all the supported formats sounds from the web ?

Yes Yes

No No

MOSQUITO Page 16/23

258067

MOSQUITO

Android Test Plan

MOSQUITO Page 17/23

258067

MOSQUITO

Android Test Plan

6. Network
This section is only applicable for applications that support network capabilities.

6.1. Protocol
5.3.1. Does the application meet the protocols implemented on the handset (like HTTP protocol) ? 5.3.2. Does the application resolve and establish a connection using an IP address and/or a domain name ? Yes Yes No No

6.2. Communication
5.4.1. Do all data sent by the application are retrieved properly, especially for the maximum amount of data for a message ? 5.4.2. Once the application has sent a data and has been terminated, is the port released ? Yes No

Yes

No No No No

5.4.3. After an error message during data transfer, Yes doest the application properly regain control ? 5.4.4. When a data packet is lost, are data correct (i.e. not corrupted) ? Yes

5.4.5. Does the application not disturb the network Yes during the test cycle (e.g. obstruction of network traffic, ) ? 5.4.6. Does the application support delayed Yes network connections ?

No

6.3. Security
5.5.1. If the handset support encryption, is it able to send secure data ? Yes No No No

5.5.2. Are sensible data (like passwords) not stored Yes locally ? 5.5.3. Are passwords not displayed when entered Yes into the application ?

MOSQUITO Page 18/23

258067

MOSQUITO

Android Test Plan

7.

Stress test

This section tests the applications running with low memory availability, interruptions and data management under stress.

7.1. Memory
6.1.1. Does the application show any signs of memory leaks, especially when adding and deleting records ? 6.1.2. If there is no more free memory and if the application does not save data, does the application run ? 6.1.3. If the storage is full and/or there is no more free memory, when you try to entry new data, does the application display an error message and are data not overwrited ? 6.1.4. For network enabled applications, when it receive a data packet larger than the available RAM, does the application not corrupt database ? 6.1.5. If the application processes data, when there are almost no more memory free, is the application able to be launched four (4) times ? If not, is an error message indicating the application cannot be launched displayed ? 6.1.6. For network enabled applications, where there are almost no more memory free, is the application able to send and receive data ? Yes No

Yes

No

Yes

No

Yes

No

Yes

No

Yes

No

7.2. Data
6.2.1. Does the application appropriately process data packets for a long time ? 6.2.2. Does the application correctly save data ? 6.2.3. When the handset is suddenly powered off, does the application not corrupt data ? Are the last saved data correctly saved ? 6.2.4. If the application accept to add records, is it able to add 30 without crash, cause the handset to reset or corrupt data ? Yes Yes Yes No No No

Yes

No

MOSQUITO Page 19/23

258067

MOSQUITO

Android Test Plan

6.2.5. When the application delete data, does it inform the user this operation will be permanent ?

Yes

No

7.3. Interruption
6.3.1. During the normal operation, does the application not corrupt data, cause the handset to reset, display an inappropriate error message or do invalid calls/operations ? 6.3.2. Does the application resume properly after 8 incoming phone calls and 8 incoming messages ? Yes No

Yes

No

7.4. User interactions

6.4.1. Tap on the screen ten times at different positions, the application should work normaly and not freeze. 6.4.2. When the application supports various ways to enter data, are all these solutions run correctly ? 6.4.3. Does the application manage correctly multi keypress ? Yes No

Yes Yes

No No

MOSQUITO Page 20/23

258067

MOSQUITO

Android Test Plan

8.

Documentation

This section tests the documentation of the application.

8.1. Documentation
7.1.1. Are all major functions of the application conform to the features described in the document ? 7.1.2. Are all the URL links in the documentation and in the application correct ? 7.1.3. Does the documentation indicate the technical support information (phone number, email address and/or web page) ? 7.1.4. Is the application provided with an User's guide in a maximum of 5 (five) PDF or HTML files ? 7.1.5. Are Help screens consistent functionality of the application ? with the Yes Yes Yes No No No

Yes Yes Yes Yes Yes

No No No No No

7.1.6. Are the screen shots consistent with User's guide ? 7.1.7. Does the documentation use the correct terminology as defined in the Users guide ? 7.1.8. Is the user able to perform essential tasks without training or help ?

8.2. Minimum requirements

7.2.1. Does the application function properly at this minimum configuration ? 7.2.2. Does the documentation include the minimum requirements : memory for the application, supported network carriers ? Yes Yes No No

8.3. Interaction
8.3.1. Advice
7.3.1. If the application support network capabilities, does the documentation indicate that other services (e.g. phone, SMS messages, ...) are blocked while the application is sending/receiving data ? Yes No

MOSQUITO Page 21/23

258067

MOSQUITO

Android Test Plan

MOSQUITO Page 22/23

258067

MOSQUITO

Android Test Plan

9.

International Applications

This section is only applicable for international applications.

9.1. Localization
8.1.1. If the language selected on the handset is not Yes supported by the application, does the application default to the correct language specified ? 8.1.2. Are the formats formatted appropriately for Yes language's country : date, time, time zone, week start, numeric separators, phone numbers, address format, name, title, weight, measure and currency ? 8.1.3. Do data entry fields accept and properly Yes display international character input ? 8.1.4. Are special characters allowed ? Yes No

No

No No No

8.1.5. Does the application respect the international Yes diversity concerning etiquette, policies and metaphors ?

9.2. Translation
8.2.1. In the application, is text content translated appropriately ? 8.2.2. In the application, is the text content not truncated ? Yes Yes No No No

8.2.3. In the document, are formated values (time, Yes date, number, currency, ) properly formatted based on the language's country ? 8.2.4. In the document, are text content and screen Yes shots translated appropriately ?

No

MOSQUITO Page 23/23