Subjective Quality Tests

To evaluate the quality of speech, audio or video systems (codec, telecommunication, television pictures, IP telephony, etc.), a subjective quality test is used. In this test, a group of human subjects is invited to judge the quality of the sequence under a predefined system conditions (distortions). There are several recommendations that specify strict conditions to be followed in order to carry out subjective tests. These recommendations are specific to the media type (speech, audio, video, or multimedia in general) in study. In this Section, we focus on how to use subjective quality tests in our method. In addition, we summarize the recommendations of ITU regarding this point. Depending on the media type, there exist some ITU recommendations that define how to carry out subjective quality tests. ITU-T Rec. P.800 [70] presents some methods for subjective determination of transmission speech quality. For audio in general, ITU-R Rec. BS.1116 [63] recommends methods for the subjective assessment of audio systems. ITU-R Rec. BS.1284 [65] presents the general requirements for subjective assessment of sound quality. How to assess subjective audio quality mixed with pictures is the subject of ITU-R Rec. BS.1286 [66]. A guide to existing recommendations that deal with audio quality assessment is provided in ITU-R Rec. BS.1283 [64]. For video quality assessment, there exist some recommendations for assessing television picture quality. These recommendations are also valid for video in general. We can cite for example, ITU-T P.910 [72]. For multimedia quality assessment, the most used recommendation is ITU-T P.911 [73]. These recommendations provide some methods for non-interactive subjective assessment methods for evaluating the one-way overall audiovisual quality of multimedia applications. For interactive applications, there exists ITU-T P.920 [74], which provides the methods recommended for subjective interactive quality tests.