2010 Signal Separation Evaluation Campaign
Overdetermined speech and music mixtures for human-robot interaction

The task was recovering the two speech signals from the mixtures of 1 female source + 1 male source + 1 music source. Therefore, the evaluation considered just speech signals: the separated signals for music were not considered in the mean values.
H. Sawada submitted separated signals which correspond to the two speech signals in the mixtures. Threfore, just 2 results are linked in the tables. On the other hand, D. H. Tran Vu submitted all the separated signals: two speech & 1 music. In the table, all 3 results are linked.

As ground truth, we have original source signals, but we do not have source image signals. Therefore, the submissions were evaluated with SIR, SAR and SDR using originals sources as "s" in bss_eval_sources.m.

All mixtures were separately recorded, without ensuring synchronization between recording and playback. The reference source signals and the mixture signals are not synchronized. Approximate synchronization was performed prior to the computation of numerical performance figures by applying a delay to the reference source signals in order to achieve maximum correlation with the mixture signal over the first channel.

For details about each algorithm, click on "Algorithm".

The audio files below are made available under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 2.0 license. The authors are M. Durkovic, M. Kleinsteuber, M. Rothbucher, and H. Shen.

Average for each microphone setup
Anechoic_H1
Anechoic_H2
Anechoic_H3
Cafeteria_H1
Cafeteria_H2
Cafeteria_H3
Office_H1
Office_H2
Office_H3
Algorithm
H. Sawada

SDR (dB)
SIR (dB)
SAR (dB)

 3.3
22.0
 3.4

 2.2
21.1
 2.3

 2.7
20.4
 2.8

-1.8
 7.8
-0.3

 0.8
12.5
 1.4

 0.8
11.3
 1.6

1.6
10.8
 2.6

 2.5
11.2
 3.7

 3.3
11.6
 4.7
Algorithm
D. H. Tran Vu

SDR (dB)
SIR (dB)
SAR (dB)

-3.6
 5.1
 0.7

-0.4
12.9
0.1

-0.3
11.8
 0.5

-3.6
 2.9
0.1

-3.5
 4.3
-1.1

-2.4
 5.9
-0.3

-3.3
 1.6
1.2

-2.7
 2.6
1.4

-2.1
 3.0
1.7


DETAILED RESULTS


Anechoic
H1C1
mix
H1C2
mix
H1C3
mix
H2C1
mix
H2C2
mix
H2C3
mix
Algorithm
H. Sawada

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 
 5.2  0.3
21.9 18.9
 5.3  0.4
src1 src2 
 4.1  3.1
24.1 21.9
 4.1  3.2
src1 src2 
 5.2  2.1
24.3 20.9
 5.3  2.2
src1 src2 
 3.9  0.2
19.2 19.2
 4.1  0.3
src1 src2 
 3.6  1.3
23.6 20.6
 3.6  1.4
src1 src2 
 3.7  0.5
24.1 19.7
 3.8  0.6
Algorithm
D. H. Tran Vu

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 src3 
-3.4 -3.1  3.6
 3.3  3.8 11.1
-0.7 -0.6  4.7
src1 src2 src3 
-8.1  0.0  1.0
-5.2 17.0  7.3
 1.4  0.2  2.9
src1 src2 src3 
-9.0  1.7  0.6
-6.6 18.1  7.4
 2.2  1.9  2.3
src1 src2 src3 
 0.6  1.9  2.2
15.8  9.8 13.8
 0.8  3.1  2.6
src1 src2 src3 
-0.1 -3.0  2.9
11.5 12.8 13.5
 0.5 -2.7  3.5
src1 src2 src3 
-0.7 -1.0  4.7
15.1 12.6 14.0
-0.5 -0.5  5.4
H3C1
mix
H3C2
mix
H3C3
mix
Algorithm
H. Sawada

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 
 4.1  0.2
18.1 19.6
 4.4  0.3
src1 src2 
 4.8  1.2
25.0 20.5
 4.9  1.3
src1 src2 
 5.1  0.5
19.5 19.6
 5.3  0.6
Algorithm
D. H. Tran Vu

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 src3 
 0.1  0.6  0.7
14.6  9.3 12.6
 0.5  1.7  1.2
src1 src2 src3 
-1.0  0.4  1.4
 5.6 16.8 11.1
 1.2  0.6  2.2
src1 src2 src3 
-0.9 -1.0  3.1
11.7 13.0 10.6
-0.3 -0.6  4.3


Cafeteria
H1C1
mix
H1C2
mix
H1C3
mix
H2C1
mix
H2C2
mix
H2C3
mix
Algorithm
H. Sawada

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 
-7.9 -0.3
-0.7  9.3
-3.6  0.7
src1 src2 
 0.5 -1.9
11.0  6.5
 1.2 -0.3
src1 src2 
 0.3 -1.5
12.0  8.6
 0.9 -0.5
src1 src2 
 2.5 -0.2
15.1 10.4
 2.9  0.6
src1 src2 
 2.3 -3.8
14.3  6.8
 2.7 -2.6
src1 src2 
 1.9  1.9
15.2 13.4
 2.2  2.4
Algorithm
D. H. Tran Vu

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 src3 
-7.8 -4.1 -3.1
-1.7 -0.4  3.5
-2.7  1.6 -0.4
src1 src2 src3 
-4.6  0.4 -5.2
-0.1  9.4 -0.2
 0.3  1.4 -0.5
src1 src2 src3 
-5.5  0.2 -5.5
 0.1  9.8 -1.3
-1.2  1.1  0.4
src1 src2 src3 
-3.4 -4.0 -0.6
 2.5  5.9  7.5
-0.1 -2.6  0.9
src1 src2 src3 
-4.2 -5.7 -2.7
 1.4  5.1  2.1
-0.4 -4.1  1.2
src1 src2 src3 
-4.9  0.9 -5.6
 1.6  9.4  1.8
-1.6  2.1 -2.5
H3C1
mix
H3C2
mix
H3C3
mix
Algorithm
H. Sawada

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 
 2.2 -0.8
 9.8  9.6
 3.5  0.0
src1 src2 
 2.7 -3.0
13.1  8.5
 3.3 -2.1
src1 src2 
 1.1  2.7
12.3 14.7
 1.7  3.1
Algorithm
D. H. Tran Vu

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 src3 
 0.3 -3.7  2.1
 7.5  6.5 11.3
 1.9 -2.4  3.0
src1 src2 src3 
-4.1 -4.1 -5.1
 2.2  6.5  1.6
-0.9 -2.8 -1.7
src1 src2 src3 
-4.1  1.6 -4.1
 1.0 11.5  3.0
-0.0  2.4 -1.4


Office
H1C1
mix
H1C2
mix
H1C3
mix
H2C1
mix
H2C2
mix
H2C3
mix
Algorithm
H. Sawada

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 
-1.3 -0.0
10.0  8.2
-0.6  1.3
src1 src2 
 2.8  3.4
11.6 12.9
 3.8  4.1
src1 src2 
 3.7  1.1
12.0  10.0
 4.7  2.1
src1 src2 
 4.3  1.4
15.1  6.9
 4.8  3.7
src1 src2 
 1.7  2.9
12.1 10.9
 2.4  4.0
src1 src2 
 3.5  1.3
14.6  7.4
 4.0  3.3
Algorithm
D. H. Tran Vu

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 src3 
-2.3 -7.8 -3.8
 1.3 -3.7  1.2
 2.6 -0.3  0.3
src1 src2 src3 
-4.0  0.4 -1.4
-0.7  7.4  4.6
 2.0  2.2  1.1
src1 src2 src3 
-4.6 -1.3 -1.4
 0.7  4.5  3.5
-0.4  1.3  1.9
src1 src2 src3 
-4.2 -3.1 -4.1
-1.0  4.8  0.1
 2.2 -1.0  0.9
src1 src2 src3 
-2.8 -0.2 -1.9
 0.7  7.2  2.6
 2.5  1.5  1.8
src1 src2 src3 
-5.3 -0.8 -5.0
-2.1  5.8 -1.2
 1.7  1.3  1.0
H3C1
mix
H3C2
mix
H3C3
mix
Algorithm
H. Sawada

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 
 4.3  1.8
16.1  6.9
 4.7  4.3
src1 src2 
 4.1  4.7
12.3 17.6
 5.0  5.0
src1 src2 
 4.1  1.0
10.5  6.4
 5.6  3.4
Algorithm
D. H. Tran Vu

SDR (dB)
SIR (dB)
SAR (dB)
src1 src2 src3 
-2.4 -1.5 -3.6
 1.5  5.1  0.2
 2.3  0.7  1.7
src1 src2 src3 
-1.4 -0.1 -3.7
 2.4  7.6 -0.7
 2.9  1.4  2.6
src1 src2 src3 
-4.7 -2.5 -4.4
-1.1  2.5 -0.4
 1.5  1.1  1.1