All algorithms addressed the source signal estimation task, which consists of recovering the original dereverberated source signals. However, since reference mono source signals were not available, the results of this task were evaluated w.r.t. the contribution of each source to the first mixture channel. Due to this mismatch between target and reference signals, the measured SDR and SAR values are irrelevant and only the measured SIR is provided below.

The measured SIR does not take human frequency sensitivity into account and may differ from the perceived SIR when the target and interference sources have different frequency ranges.

For details about each algorithm, click on the algorithm number.

For summary results, see
E. Vincent, S. Araki and P. Bofill, "The 2008 Signal Separation Evaluation Campaign: A community-based approach to large-scale evaluation", in Proc. Int. Conf. on Independent Component Analysis and Signal Separation, 2009.

The mixture audio files are licensed for research use only by their authors Kenneth Hild (Iliad), Hiroshi Sawada (rooms 4 and 5), Mads Dyrholm (rooms 1, 2, 3, C and O) and Lucas Parra. The music used for recordings in rooms 1, 2 and 3 was taken from "Germ Germ" by Das Böse Ding and has kindly been approved for public presentation by Jan Klare of Das Böse Ding in the name of research. See Lucas Parra's BSS page for the original dataset. All other audio files are made available under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 2.0 license.

(1) The mixture signals were produced by separately recording the spatial image of each source over all microphones then summing the spatial images of all sources over each channel.

(2) All signals were separately recorded, without ensuring synchronization between recording and playback. The reference source spatial images and the mixture signals are not synchronized. Approximate synchronization was performed prior to the computation of numerical performance figures by applying a delay to the reference source spatial images in order to achieve maximum correlation with the mixture signal over the first channel. Therefore, the measured SIR is inherently less accurate than in case (1).
If often happens over 4-source mixtures that the 4th source is not predominant in any of the estimates, while another source is predominant in two different estimates. In this case, the algorithm failed to recover all sources as requested, which is reflected by a large negative SIR for the 4th source.
It also happens that some estimates correspond to different sources at different frequencies. In this case, the permutation of the sources estimated by maximizing SIR may be perceptually wrong, since the SIR does not take human frequency sensitivity into account. Nevertheless, this does not affect much the mean SIR.