In recent years Voice over IP (VoIP) telephone networks have been coming into increasing usage. Along with these networks, various test equipment has become available to test and monitor the performance of these systems. Much of the testing effort has been focused on the network, and the telephone itself is at times overlooked. However, the VoIP telephone is an important part of the signal path and can greatly influence the quality of conversations. The use of poorly performing telephones can result in an unsatisfactory user experience, even when used on an ideal network. Common problems that can be caused by the telephone include interoperability problems, poor intelligibility, noise, distortion, latency, and echo.

Many VoIP telephones are being developed and manufactured by companies that are new to the field of voice communications. At times, the knowledge gained through years of experience by the traditional telephone industry has not been applied to the development of VoIP telephones. In addition, the packetization inherent in VoIP introduces additional issues for the VoIP telephone designer.

VoIP Performance Tests

The use of a well designed VoIP telephone is essential for a positive user experience on the VoIP network.
Some aspects of a VoIP telephone’s performance that can be tested are:

a) Network Signaling Protocol

The signaling protocol (eg SIP, H.323) can be tested for compliance with the appropriate standard. Protocol analysis tools and interoperability tests are used to verify compliance.

b) Acoustic Performance

The performance of the acoustic to electrical and electrical to acoustic conversion can be measured. Tests are performed by the measurement of various electro-acoustic parameters.

c) Response to Network Impairments

The telephone is tested to determine how well it handles packet loss, network jitter and other network disturbances. Tests can be performed using mean opinion score (MOS) prediction algorithms (such as the ITU-T P.862 PESQ) and a network impairment simulator.

Acoustic Performance Tests

The Telecommunications Industry Association (TIA) has established standards for the acoustic performance of digital telephones. TIA-810-A (currently being revised to TIA-810-B) specifies requirements for narrowband telephones, and TIA-920 specifies requirements for wideband telephones.

Some of the acoustic tests that affect user’s perceived quality are listed below:

Frequency Response Tests

These tests measure the frequency dependence of the acoustic to electrical and electrical to acoustic conversion. The frequency response is measured for Send, Receive, and Sidetone (except for handsfree) modes. The measurement of frequency response is important to ensure that the telephone does not inappropriately emphasize or attenuate certain frequencies.

Loudness Tests

A weighted average of the frequency response is calculated to provide a single number representing perceived loudness of the telephone. The Send Loudness Rating (SLR), Receive Loudness Rating (RLR) and Sidetone Masking Rating (STMR) are calculated. The nominal loudness ratings are specified in the network loss plan to ensure that loudness is consistent between different telephones on different connections and that there is adequate head room to accommodate peak signal levels.

Volume Control Tests

The volume control is tested to determine the range of loudness provided. This is not only a performance requirement, but also a legal requirement in many countries; for example in the U.S.A. the Hearing Aid Compatibility Act of 1988 (HAC Act) requires a certain range of volume control and magnetic output to ensure reasonable access to telephone service by persons with hearing disabilities.

Noise & Distortion Tests

These tests are performed to measure the amount of noise and distortion generated by the telephone. Several different stimulus levels and frequencies are typically tested. Noise and distortion can fail due to a number of causes: poor codec implementations, bad or improperly installed transducers, faulty electrical components, poor connections, handset plastics flaws, etc. Good distortion performance indicates that the VoIP phone can reliably reproduce signals.

Noise and Single Frequency Interference Tests

This test detects spurious signals in the noise generated by the telephone that might be perceived as a tone. This test is performed to verify the generated noise is minimal and because telephone users find that a tone is more objectionable than random noise.

Weighted Terminal Coupling Loss Test

This test measures how much of the receive signal that is sent to the telephone is re-transmitted to the network. Because of delay inherent in VoIP telephony, a re-transmitted signal will be heard at the far end telephone as an echo. The echo frequency response is measured and the weighted terminal coupling loss (TCLw) is calculated as a measure of the echo from the telephone.

Latency Tests

The time delay of the telephone is measured. The delays in the VoIP telephone add to the network delays and can have a significant affect on the call quality. Generally VoIP telephones should add minimal delay; TIA-810-A requires that the send latency be less than 35 ms and the receive latency be less than 65 ms.

Acoustic performance tests are typically done with a closed network to isolate the IP phone from the effects of network variations. The acoustic tests are also performed using the G.711 codec for narrowband operation and linear 256 kbits / sec PCM codec for wideband operation. These tests are not intended to compare or verify the performance of different voice codecs. To test a codec implementation, test vectors and other performance criteria are specified in the standards defining the codec. For comparison of the performance of different codecs or network conditions, MOS estimation algorithms such as PESQ are often used.

The acoustic tests should be performed for handset, handsfree, and headset operation. The standards’ requirements are different for each mode of operation.

Test Configurations
There are two approaches to performing acoustic performance tests on digital telephones.

The first, preferred, approach to testing is “direct digital” (see Figure 1) where the test equipment generates and analyzes digital signals directly without conversion to analog. Test equipment designed specifically for testing VoIP telephones is required, but the results obtained are more accurate because the reference codec is not used.

When using the direct digital method, a telephone call is established between the telephone under test and the test equipment. This requires that the test equipment support the protocol used by the telephone. This approach works well for telephones using common protocols such as SIP and H.323. For cases where the telephone uses a proprietary protocol not supported by the test equipment, it is sometimes possible to bypass call establishment and start the Real Time Protocol (RTP) used for audio transport by placing the telephone into a manufacturing test mode. Most VoIP telephones use RTP for audio transport.

The second, alternative, approach is use to use a “reference codec” (see Figure 2) to provide an analog interface to the telephone. This has the advantage of allowings testing to be performed using general purpose instruments or traditional telephone test equipment. When using the reference codec approach, it is important to be aware of the effect of time delays in the reference codec and telephone. The combination of the reference codec and telephone can result in more than 100 ms between application of a test signal to system, and the output signal being available for measurement. Most traditional telephone test equipment was not designed with the expectation that this signal path delay would be present, and the equipment must be configured to account for it.

The reference codec is intended to have ideal characteristics, but a physical implementation is never perfect and the resulting measurements include the effects of both the device under test and the reference codec. It can be difficult to determine how much effect the reference codec has on the readings in a given test setup. Latency tests are often not possible with the reference codec method because the exact delay of the
reference codec is unknown.

Direct Digital test equipment is designed for testing digital telephones and takes into account the signal delay through the telephone. Time alignment of the measured signal can be used for analysis to compensate. The internals delays of the test equipment can be determined to allow measurement of the telephone’s latency.

For either the reference codec or direct digital methods, an acoustic fixture is required to interface to the telephone’s speaker and microphone. The type of fixture used can vary depending on whether handset, headset, or handsfree operation is being tested.

The recommended acoustic interface for headset and handset testing is a Head and Torso Simulator (HATS). The HATS simulates the shape of an average human head and torso and includes a mechanism to position and hold a handset. The HATS is equipped with one or two ear simulators that simulate human ear characteristics and a mouth simulator to apply stimulus to the telephone’s microphone. For Handsfree operation, a different test fixture with a free field microphone instead of an ear simulator is typically used.

Many of the acoustic tests must be performed in an quiet environment free from acoustic reflections. The telephone and test fixture are often placed in an anechoic chamber to achieve these conditions.

Microtronix Systems Ltd., a worldwide telephone test system provider since 1972, provides VoIP manufacturers and developers with a test solution to evaluate the acoustic performance of an IP phone (Desktop, WiFi, WLAN). The unique feature that Microtronix provides is Direct Digital Generation with the IP Phone. This allows direct digital communication with the IP phone without analog conversion. High speed, accurate, and repeatable standards-compliant results allows you to reduce development time, costs and increase production.

The IP test solution can measure Send / Receive Latency (the amount of time the phone requires to encode and decode audio). This ensures the phones’ delay does not affect the quality of the call. Tests such as Send / Receive, Frequency Response and Loudness Rating, Echo and Weighted Terminal Coupling Loss, Distortion and Noise are provided by the test system. Microtronix provides applications for handset, handsfree, Analog Telephone Adaptors (ATA), headsets and custom applications. Microtronix IP test solution can measure the latency (the amount of time the phone requires to encode and decode audio).

Microtronix supports protocols such as Session Initiated Protocol (SIP) IETF RFC 3261, H.323 (ITU-T H.323 version 4), Session Initiated Protocol (SIP) IETF RFC 3261, Wideband and directly over RTP (Real-time Transfer Protocol). The system architecture is also designed to implement custom protocols. A pre-programmed VoIP test suite- for specifications such as TIA/EIA-810-A and TIA-920 standard or custom test suites are also available.

Microtronix test equipment is being used globally in countries such as China, Thailand, United Kingdom, Malaysia, and the United States. To date, Microtronix has test equipment in over 40 countries worldwide. Many contract manufacturers from small to large use Microtronix test equipment to ensure the products they supply to their principal companies meet the quality and volume goals.