Stereo Audio Source Separation Evaluation Campaign
Results

For summarized results and references about the algorithms, see
E. Vincent, H. Sawada, P. Bofill, S. Makino and J.P. Rosca, "First stereo audio source separation evaluation campaign: data, algorithms and results", in Proc. Int. Conf. on Independent Component Analysis and Signal Separation, 2007.

Instantaneous mixtures
Female
sim1 sim2 sim3 sim4
mix
Male
sim1 sim2 sim3 sim4
mix
Nodrums
sim1 sim2 sim3
mix
Wdrums
sim1 sim2 sim3
mix
Algorithm 1(1)
D. Barry
(1 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  7.0   5.0   4.1   5.5
  8.5   6.0   5.8   7.2
18.5 16.1 11.2 15.0
  8.9   6.6   4.2   6.5
sim1 sim2 sim3 sim4
  8.4   2.6   4.6   4.6
  9.4   5.1   6.0   5.7
22.4   5.8 11.6 14.0
12.2   1.5   5.5   5.5
sim1 sim2 sim3
10.0   0.6   2.6
10.6   8.4   5.4
24.3   9.3 12.8
16.4  -1.5   0.6
sim1 sim2 sim3
 -6.1  -0.7 10.9
  9.1   4.8 11.7
 -0.7   2.4 24.7
 -4.2  -2.8 16.5
Algorithm 2
P. Bofill
(5 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  3.6   6.2   4.0   3.4
  3.8 10.3 11.6   6.1
18.0 10.6   7.8 12.6
13.5   7.6   6.0   9.7
sim1 sim2 sim3 sim4
  3.8   1.5   4.8   3.2
  3.9   8.3 11.7   6.7
20.6   2.6   9.6 11.2
16.2   2.7   6.5   8.6
sim1 sim2 sim3
  8.3  -3.2   6.3
  8.6 11.3   8.3
20.0  -2.9 15.0
20.1   4.8 14.1
sim1 sim2 sim3
  3.1   5.7   6.9
  8.3   8.4   7.0
  4.6 14.2 33.8
  6.3   5.8 25.2
Algorithm 3
A. Ehmann
(5 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
10.3   5.9   4.6   7.5
17.1 12.6 11.9 11.7
22.3 14.0   9.4 19.9
10.5   6.0   5.2   8.0
sim1 sim2 sim3 sim4
11.1   1.3   4.1   6.6
18.3   8.9 14.5 11.9
23.9   6.2   7.6 17.3
11.5   0.7   5.9   6.3
sim1 sim2 sim3
15.1  -6.8   3.7
20.0   8.9   5.6
25.8  -4.9 18.1
17.3  -0.6   3.6
sim1 sim2 sim3
  7.2   3.0 20.5
15.8   4.7 31.1
14.2 15.1 27.7
  8.9   2.1 21.8
Algorithm 4
V. Gowreesunker
& A. Tewfik
(10 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  2.7   4.3   3.1   3.8
  2.9   7.3   7.4   6.1
18.9   9.7   7.7 15.9
  8.5   4.3   2.7   5.7
sim1 sim2 sim3 sim4
  2.8   0.5   3.9   3.4
  2.9   5.7   8.0   6.1
20.2   2.1 10.4 15.0
  9.0  -0.4   3.8   4.2
sim1 sim2 sim3
  5.7  -2.2   3.4
  6.1   8.0   5.0
27.1  -2.0 18.6
10.9   1.3   4.0
sim1 sim2 sim3
  4.8   3.3   8.0
  6.2   4.8 10.0
12.1 17.2 25.1
  6.8   2.2 12.3
Algorithm 5(2)
M. Kleffner
(10 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
-20.5 -26.8 -22.0 -21.8
-19.2 -25.7 -20.4 -20.6
18.6 12.6   9.2 14.3
  5.8   7.2   6.2   6.5
sim1 sim2 sim3 sim4
-19.2 -29.6 -26.6 -20.5
-18.1 -26.6 -25.2 -18.7
19.9   4.6 10.9 11.9
  6.6   3.6   6.3   4.9
Algorithm 6(2)
N. Mitianoudis
(3 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
-17.2 -21.4 -18.9 -18.3
-15.9 -18.7 -14.7 -16.4
19.0   6.3   7.9 16.9
  6.1   4.3   0.5   4.1
sim1 sim2 sim3 sim4
-17.8 -23.5 -17.9 -17.4
-17.0 -17.4 -14.8 -15.2
22.5   2.0 10.8 16.0
  8.1  -0.4   2.1   3.5
sim1 sim2 sim3
-11.1 -25.1 -10.2
 -9.8 -13.5  -6.2
13.3  -7.5 17.5
  8.5   2.5   1.4
sim1 sim2 sim3
-12.1 -10.9  -8.5
-10.6  -7.6  -8.4
16.6 15.9 26.7
  6.5   2.7 20.1
Algorithm 7
H. Sawada
(9 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  9.0   5.6   4.0   6.2
15.0 14.6 11.7   9.0
20.8 11.2   7.7 20.3
  9.1   6.3   4.8   7.0
sim1 sim2 sim3 sim4
10.8   0.5   5.0   6.0
17.3   9.9 13.1   9.8
22.7   4.0   9.5 17.5
11.2   1.1   5.8   5.7
sim1 sim2 sim3
15.5  -0.8   4.7
20.9 15.6   6.9
23.5   2.8 18.3
17.8   3.1   5.2
sim1 sim2 sim3
  9.1   5.1 17.5
16.9   8.5 30.7
17.1 18.0 25.7
10.5   4.5 18.3
Algorithm 8
E. Vincent
(5 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
13.3   6.8   5.8   8.4
26.6 11.1 11.4 18.2
17.4 16.8 12.0 13.0
15.4   7.1   6.1 10.0
sim1 sim2 sim3 sim4
16.4   3.2   6.1   7.9
28.3   7.4 10.9 20.0
20.4   8.4 13.9 12.0
18.8   2.5   6.3   9.6
sim1 sim2 sim3
22.2   2.7 16.8
32.1 10.2 27.6
27.8   7.8 22.0
24.1   3.2 18.5
sim1 sim2 sim3
 -0.6   3.1 28.3
  8.8   4.5 46.3
  1.9 17.0 29.8
  7.7   2.5 34.1
Algorithm 9
M. Xiao
(2 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  9.1   2.1   0.4   3.7
23.1   8.3   8.7 10.3
14.7   9.6   3.6 10.9
10.4   0.8   0.6   3.1
sim1 sim2 sim3 sim4
12.6  -0.8   1.6   3.6
27.6   4.2   9.6 12.6
17.6   3.1   6.2   9.6
14.2  -4.2   0.9   3.7
sim1 sim2 sim3
14.0  -5.3   8.8
29.8   1.4 19.8
19.3  -7.1 14.0
15.6  -7.3   9.7
sim1 sim2 sim3
 -0.7   3.0 26.0
  8.9   8.3 43.9
  2.2   9.9 33.0
  1.1   1.4 27.1
Algorithm 10
M. Xiao
(2 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  8.1  -1.5  -2.1   2.5
27.2 18.0 18.6 20.7
11.1   3.2  -0.5 5.4
11.1   1.5   5.0   6.1
sim1 sim2 sim3 sim4
12.3  -8.0  -0.5   2.3
29.4 15.0 21.4 23.1
15.2  -3.5   1.9   4.4
15.5  -1.1   5.4   7.7
sim1 sim2 sim3
14.2 -13.6 13.5
29.9   5.8 32.3
19.3 -10.6 14.2
15.8   3.4 22.2
sim1 sim2 sim3
 -2.8   3.6   8.1
  6.4 20.9 14.3
 -0.4   5.6 25.9
  8.2   8.4   8.0
Algorithm 14(3)
R. Weiss
and M. Mandel
(20 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  3.6   0.3  -3.4   0.2
  5.5   0.3 12.4   0.2
  5.2  -5.7  -4.7  -7.2
  5.6   0.1   9.9  -0.7
sim1 sim2 sim3 sim4
  1.5  -8.5   0.4   0.3
  1.7 10.1   0.6   0.5
  3.1  -9.9  -5.6  -4.8
  3.2 11.7   1.5  -0.6
sim1 sim2 sim3
  4.0  -9.2  -2.0
  4.5   5.0   5.8
15.8 -13.6  -3.0
  6.5   8.4   0.1
sim1 sim2 sim3
-11.8  -9.8   4.0
  0.0   6.2   4.4
-18.8 -10.7 16.8
  2.4   3.5   7.1


Synthetic convolutive mixtures with 5 cm microphone spacing
Female
sim1 sim2 sim3 sim4
mix
Male
sim1 sim2 sim3 sim4
mix
Nodrums
sim1 sim2 sim3
mix
Wdrums
sim1 sim2 sim3
mix
Algorithm 11
S. Araki
(1 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  5.6   3.7   3.2   3.0
  8.4   6.6   7.2   5.0
10.6   7.1   4.9   6.8
  8.7   5.9   6.0   5.3
sim1 sim2 sim3 sim4
 -0.3   2.1   2.5   0.1
  7.1   2.6   7.2   4.0
  1.1   7.1   3.7   4.9
  4.4   2.1   5.2   1.3
Algorithm 14(3)
R. Weiss
and M. Mandel
(20 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  0.4   0.2  -0.4   0.4
  0.4   6.2   5.4   0.5
 -2.2  -4.1  -5.2  -3.3
  6.7 18.7 18.9   7.2
sim1 sim2 sim3 sim4
  0.6   0.9   2.0   0.5
  0.7   1.8   3.9   0.5
 -2.1  -5.3  -1.8  -2.6
  8.0 18.2 18.2   7.9
sim1 sim2 sim3
  0.5   3.3   0.8
  0.7 12.3   0.9
 -6.2   2.2  -1.1
11.5 25.0   9.8
sim1 sim2 sim3
  2.1   0.4   0.8
  2.9   0.6   1.1
  1.8  -5.1  -3.9
19.1   8.9 16.5
Algorithm 15
H. Sawada
(40 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  5.8   3.9   1.4   1.6
10.1   8.0   8.6   3.3
10.6   7.1   1.6   4.7
  8.1   5.9   6.0   2.8
sim1 sim2 sim3 sim4
 -0.9   2.2   2.4  -0.4
  7.4   2.8   8.9   4.0
  0.8   6.4   3.9   5.1
  4.7   2.1   5.4   1.5
sim1 sim2 sim3
  0.3   1.9  -1.8
  3.8   2.4   8.3
  0.7 10.8  -1.8
  4.9   9.7 11.4
sim1 sim2 sim3
  1.5 -11.7   0.8
  1.8  -2.5   1.0
  7.6 -10.0 15.7
14.5 15.8   6.5


Synthetic convolutive mixtures with 1 m microphone spacing
Female
sim1 sim2 sim3 sim4
mix
Male
sim1 sim2 sim3 sim4
mix
Nodrums
sim1 sim2 sim3
mix
Wdrums
sim1 sim2 sim3
mix
Algorithm 14
R. Weiss
and M. Mandel
(20 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  1.5   1.4   1.6   1.2
  2.4   3.4   4.8   3.0
  0.2  -2.6  -1.5  -2.9
  6.7   9.1   9.2   8.4
sim1 sim2 sim3 sim4
  2.1   0.6   2.8   0.6
  3.6   0.8   4.6   3.0
  0.2  -3.3   1.5  -3.6
  9.0   7.1 10.7   8.5
sim1 sim2 sim3
  1.6   2.8   2.2
  4.1   4.4   6.1
 -0.4   3.6   0.1
  9.2   8.7 10.6
sim1 sim2 sim3
  2.6 -10.0   0.7
  2.7  -3.6   1.0
11.4  -5.6  -4.9
16.9 18.0 13.9
Algorithm 15
H. Sawada
(40 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  6.4   5.3   3.3   2.7
10.2   9.9   8.9   5.6
12.1   9.0   5.3   7.3
  8.7   7.6   5.9   4.7
sim1 sim2 sim3 sim4
 -0.2   1.2   1.6  -3.0
  6.7   1.8   6.6   0.5
  1.4   1.7   2.8   5.2
  4.1   2.3   4.6   3.1
sim1 sim2 sim3
  3.0   1.1  -1.8
  5.5   3.1   4.1
  8.4   2.4  -1.7
  4.6   9.0   9.3
sim1 sim2 sim3
  4.5 -12.7   0.6
  4.9  -3.1   0.9
14.6 -10.7   4.2
16.3 13.7   6.9


Live recordings with 5 cm microphone spacing
Female
sim1 sim2 sim3 sim4
mix
Male
sim1 sim2 sim3 sim4
mix
Nodrums
sim1 sim2 sim3
mix
Wdrums
sim1 sim2 sim3
mix
Algorithm 11
S. Araki
(1 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  2.8  -0.4   1.7   4.8
  6.0   4.0   2.5   9.3
  4.6  -1.9   5.4   8.6
  5.9   6.5   4.0   6.5
sim1 sim2 sim3 sim4
  3.3   0.5   3.7   4.3
  6.5   1.2   9.2   8.6
  6.0   2.3   4.5   7.2
  5.0   1.3   7.4   6.6
Algorithm 12
Y. Izumi
(1 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  1.4   1.3   1.3   1.6
  2.1   1.9   1.5   2.0
 -1.1  -0.9   2.7   2.1
  6.4   6.6   6.4   6.9
sim1 sim2 sim3 sim4
  1.7   1.1   2.1   2.0
  2.4   1.5   2.4   2.3
 -0.1  -1.5   4.6   5.1
  6.7   5.9   7.1   6.8
Algorithm 13(2)
T. Kim
(1 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim3 sim4
-20.6 -18.3
-16.9 -14.1
  1.4   1.8
  7.4   4.5
sim3 sim4
-23.4 -19.1
-20.8 -16.2
  4.2   4.1
  6.3   6.4
Algorithm 14(3)
R. Weiss
and M. Mandel
(20 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  0.9   0.2   1.7  -0.5
  1.2   0.2   2.3   9.4
 -4.2  -6.0  -1.5  -3.0
11.9   4.7 17.1 23.6
sim1 sim2 sim3 sim4
  0.9   0.2   1.6   2.2
  1.1   0.2   6.9   5.7
 -2.9  -6.0  -1.4  -1.1
  8.8   2.2 15.7 15.7
sim1 sim2 sim3
  6.0  -2.7   0.2
  6.7   5.6   0.4
  9.2  -7.6  -9.9
20.4 19.6   7.9
sim1 sim2 sim3
  2.2   1.3   2.1
  3.2   1.6   9.9
  1.0   3.7  -0.1
10.2   6.3 17.1
Algorithm 15
H. Sawada
(40 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  2.6  -0.8   1.7   4.2
  6.5   4.5   2.8 10.4
  4.4  -2.2   5.4   7.7
  5.6   6.3   3.7   6.1
sim1 sim2 sim3 sim4
  3.0   0.4   3.2   3.9
  7.4   1.4 10.5   9.9
  5.6   1.7   4.0   6.7
  4.6   0.7   7.1   6.2
sim1 sim2 sim3
  4.4  -2.6  -4.6
  5.0 11.4   5.5
15.5  -3.2  -5.6
11.6   9.7   6.7
sim1 sim2 sim3
  2.8   3.9   4.1
  7.0   7.2   7.8
  3.9   8.2   6.1
  7.7   7.7   8.3


Live recordings with 1 m microphone spacing
Female
sim1 sim2 sim3 sim4
mix
Male
sim1 sim2 sim3 sim4
mix
Nodrums
sim1 sim2 sim3
mix
Wdrums
sim1 sim2 sim3
mix
Algorithm 13(2)
T. Kim
(1 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim3 sim4
-16.8 -21.6
-12.7 -17.2
  1.7   0.4
  6.5   5.7
sim3 sim4
-19.5 -18.2
-16.4 -15.5
  4.4   5.1
  5.0   5.9
Algorithm 14
R. Weiss
and M. Mandel
(20 min/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  1.6   0.7   3.4   1.3
  3.1   3.8   4.9   3.9
 -0.4  -3.0   4.2  -1.9
  5.9   5.7   8.1   7.7
sim1 sim2 sim3 sim4
  1.7   1.4   3.0   2.0
  3.7   3.9   4.4   4.5
 -0.4  -1.9   3.4   0.2
  5.8   6.6   7.4   8.0
sim1 sim2 sim3
  5.3   1.7  -1.7
  6.6   4.4   6.0
  9.8  -0.5  -4.6
11.0   6.8   9.1
sim1 sim2 sim3
  4.0   2.0   2.6
  8.0   4.4   4.6
  4.2  -0.4   1.1
10.5   8.1   9.2
Algorithm 15
H. Sawada
(40 s/mix)

SDR (dB)
ISR (dB)
SIR (dB)
SAR (dB)
sim1 sim2 sim3 sim4
  4.5   3.8   7.4   3.3
  9.1   8.0 13.1   6.2
  8.0   7.1 12.2   7.4
  6.3   5.4   9.5   4.7
sim1 sim2 sim3 sim4
  3.0   1.5   5.2   2.3
  7.9   4.7   9.0   6.5
  5.1   2.6 11.0   4.7
  4.7   2.7   6.1   4.6
sim1 sim2 sim3
  4.3   5.1  -4.0
  5.7 13.6   3.7
10.1   8.1  -6.2
10.2   7.8   7.5
sim1 sim2 sim3
  4.0   4.5   6.5
  6.8 11.6 10.7
  8.3   7.0 12.2
  7.4   8.1   8.1

(1) Negative SIR/SAR values due to strong time-localized interference within the last 100 ms of each estimated source image signal.
(2) Large negative SDR/ISR values due to incorrect scaling of the estimated source image signals.
(3) Some negative SIR values due to missing sources among the estimates, replaced by other sources or almost silent signals.