Cellular Phone Conformance Testing

Today, vast numbers of mobile phones are in use around the globe. 2006 saw in well excess of 2 billion subscribers connected and over 1 billion phones manufactured. When users buy phones they expect that the system will work. They are not interested in the reasons why there may be problems, often blaming the phone itself. Poor service, for whatever reason will result in users choosing a different network, and adding to the rate of churn. Ensuring that cell phones operate correctly when deployed is no easy task. Testing of the design is required at all stages of the development. It is necessary to check the hardware, and software. Once the cell phone is assembled, full testing of the complete "system" is required, testing it against its requirements and specification. Then prior to deployment it must undergo formal testing and be "approved" before it can be used. Dependent upon the type of cellular network, i.e. GSM / UMTS, or CDMA (cdmaOne / cdma2000) this formal "approval" may take one of two forms:

Conformance test Interoperability test

Once the formal approval has been gained, the cell phone can then be manufactured in quantity, sold and deployed on the cellular network. In many respects cell phone conformance test and interoperability test perform the same function but they have some significant differences between them. A conformance test tests for conformance to a particular specification, whereas an interoperability test checks that the phone will work on a given network. Both conformance test and interoperability tests have the advantages and disadvantages. However a conformance test is required for GSM, UMTS cell phones and generally interoperability tests are required for phones for many of the CDMA networks.

Conformance and interoperability tests

To ensure that a mobile phone meets its required standards it has to undergo a variety of types of test. These are often categorised into different areas. In order to undertake these tests different test house may be required. 1. Basic safety testing This is a form of test that every piece of equipment, whether mobile phone or otherwise has to undergo to ensure that it is intrinsically safe to use and no injury will be inflicted for example from sharp edges, etc.. 2. SAR, Specific Absorption Rate This test involvesmeasuring the amount of radio frequency power that a human head will absorb when the cell phone is

3.

4.

5.

6.

transmitting. The test uses an anatomically correct model of the human head. Inside the model temperature sensors are set up to measure the temperature rises to ensure that the heating effects caused by the cell phone fall below acceptable limits. Protocol testing One of the major areas of cellular conformance testing is the protocol testing of the cell phone. With the complicated protocols used in mobile phones this is a critical area. If the phone protocol software operates incorrectly then it could result not only in problems experienced by the phone, but also on the network. In view of the complexity of the protocols that are used this testing can be very involved. Specialised network simulators are used. These testers emulate a variety of network entities, i.e. base stations or Node B's (in the case of UMTS), RNCs (Radio Network Controller and the like. In this way a host of scenarios from registration to handover, and in fact any situation that can be encountered can be simulated. RF testing Conformance testing also includes testing of the RF signal. Many measurements of the transmitter and receiver performance are undertaken in a variety of areas such as the out of band emissions. Measurements of the Radio Resource Management (RRM) are undertaken to ensure that the control capability of the phone is operating correctly. There are for instance very tight limits on the control of the transmitter output power to ensure that the cell phone radiates only as much as is needed under any given conditions and noise in the phone bands is reduced to the minimum level. To achieve this testing a protocol tester is often used to control the phone and set up the relevant scenarios. In addition to this an RF measurement and generation equipment is required. This is often in the form of additional signal generators, power meters, analysers, noise generators, etc. To check operation of the phone with multi-path and fading, special fading simulators are required. SIM card testing Another very important area of cellular conformance or interoperability testing is the operation of the SIM card, or in the case of UMTS the USIM. As SIMs are interchangeable between phones it is necessary to rigorously check the interface. It is also vital to check the security aspects of the operation of the SIM, as lapses in security could compromise elements of the network security. To undertake this testing a SIM simulator (or USIM simulator) is required. This simulator emulates the operation of the SIM, and tests on the phone can then be run using a protocol tester to set up the variety of scenarios that are needed. Audio tests Finally audio checks of the cell phone are undertaken. These check the correct operation of the audio aspects of the cellular phone, both in terms of the microphone and the earphone. Checks of audio levels, quality and much more are measured using a variety of audio equipment to ensure they conform to the requirements laid down.

Test cases
It is obviously necessary to ensure that each stage in the testing is repeatable regardless of the test equipment is used and the organisation performing the testing. To achieve uniformity a large number of what are termed "test cases" are defined. These may be expressed in a number of ways, but typically they may be in prose, or they may be in a form of computer notation called TTCN (Tree and Tabular Combined Notation). In the latter format they can often be compiled directly into code that can be run on a given test system. For any given standard, i.e. GSM UMTS, cdma2000, etc there are many hundreds of test cases that need to be run. They are prepared under the auspices of the governing body, e.g. 3GPP for GSM and UMTS. Manufacturers of conformance test equipment are then able to take these test cases and convert them to run on their test equipment. Often the test cases must be ratified on a given test system before they can be used towards approval of a phone.

GSM / UMTS conformance testing

The process for conformance testing for GSM and UMTS is well defined and operates well. The conformance test approval process is run by the industry itself and is effectively self regulating. However the overall GSM / UMTS conformance testing process works well, and the fact that all phone designs have to pass the conformance test process means that the number of problems discovered with phones is very low. Additionally as all phones conform to the same standards, it is possible to roam from one network in one country to another network in another country with no problems, provided that there are agreements betweent he oerpators. This has been one of the major reasons fort he global success of GSM, and much of this can be attributed to the conformance test process. For GSM these test cases were written in prose, and they described the test itself, the setup conditions as well as the applied stimuli and of course the pass and fail criteria. It was then possible for each test equipment manufacturer to implement these tests on their equipment. However to ensure that the tests are a faithful implementation of the original intent, a validation and certification process has been set up. The conformance test cases were originally defined by ETSI (European Telecommunications Standards Institute) for GSM, but they are now controlled by 3GPP (Third Generation Partnership Project). Once the manufacturer is satisfied that the test operates satisfactorily he provides it to an independent validation organisation that will test it and for conformance with the original test case. Assuming that the test case satisfactorily passes this process then it is presented to an industry body for certification. Once it has achieved this status then it can then be used for formal handset testing and certification.

The organisation that has overall control of the test cases for GSM and UMTS is 3GPP. However changes are handled by the GERAN (GSM Enhanced Radio Access Network) working Group for GSM and a group known as the T1 group for UMTS. The validation and approval of the implemented test cases is then handled by the GCF (Global Certification Forum). The North American version of GSM running in the 1900 MHz band is often referred to as PCS (Personal Communications System) and there is another allocation at 850 MHz. A group known as PVG (PCS Validation Group) handles the approvals and their results are ratified by PTCRB (PCS Type Certification Review Board). Phones are then tested against the test cases which if successful are certified by CTIA. To achieve CTIA certification, it is necessary for phones to be tested in CTIA approved laboratories. As might be expected, experience gained on GSM has been reflected into improvements for the new third generation UMTS system. One of the main changes is that the protocol test cases, are initially written in prose and then converted into TTCN (Tree and Tabular Combined Notation) to be made available to the industry. This language enables the test cases to be compiled into a format that can be run directly on the target test equipment. This approach saves time for the industry as a whole and reduces costs because generating the test cases is far easier to achieve. The main advantage is that it gives far more consistency across the industry as tests are no longer open to the same level of interpretation that they were before. This also saves time in the validation process. The TTCN test cases have been prepared by a team of industry experts based at ETSI working on behalf of 3GPP. They prepared the basic TTCN code that was reviewed within the 3GPP community using email reflectors. By reviewing the software in this way at the beginning of the process, the individual test cases do not need to be reviewed each time they are submitted by a test equipment manufacturer for validation. To test a new handset, the manufacturer generally approaches a qualified test house who will possess a variety of different test systems. They will then run the required certified test cases and be able to present a case for a handset being suitable for use on the available networks.

CDMA interoperability testing

The approach for approvals testing for cdmaOne / CDMA2000 cell phones is that of an interoperability test. Here the emphasis of testing is for interoperability with the network rather than conformance with a specification - the approach that is adopted by the GSM / UMTS fraternity. For CDMA2000 the standards are written by 3GPP2. This is the equivalent of 3GPP, but for the CDMA standard IS-2000. These standards are then published by the relevant standards body for a particular area. In North America this is TIA (Telecommunications Industry Association), for Japan it is ARIB (Association of Radio Industries and Businesses) etc..

In North America, testing is carried out under the auspices of either CDG (CDMA Development Group) or CTIA. Documents written by CDG reference the 3GPP2 and TIA standards. Testing is then conducted in three stages, namely Stages 1, 2, and 3. Stage 1 testing verifies the RF performance of a system, checking parameters such as receiver sensitivity, performance under fading conditions and the like. Stage 2 is what is known as the Cabled Interoperability Testing. Here the protocols of the mobile handset are checked against a base station to ensure the mobile performs in the correct manner under a variety of conditions such as call set-up, handoff, and the like. Finally there is the Stage 3 testing. This is undertaken by an operator and consists of running the mobile on a live network to ensure that it works over the air. This may include what is termed a drive test where the mobile is driven around a live network and its performance checked under real conditions. The testing carried out under CTIA auspices is undertaken by approved laboratories. They use test equipment to simulate the network, and tests similar to those in CDG Stages 1 and 2 are employed. Once mobile phones successfully pass the CTIA tests they are given an approval certificate. Different test requirements are placed upon handset manufacturers by the various network operators, and therefore phones may undergo a variety of tests under either or both the CDG and CTIA auspices.