TMO optimization for HDR compression

 
Logo Not found  TMO optimization for HDR compression

contact: D. GommeletA. RoumyC. GuillemotM. RopertJ. Le Tanou

This webpage brings additional results to the paper "Rate-Distortion Optimization of a Tone Mapping with SDR Quality Constraint for Backward-Compatible High Dynamic Range Compression" [1] submitted to icip 2016. The following figure recalls the scheme used to encode the HDR content:

Not Found AtriumNight.png

To remind how the test is done, first the original HDR image is transformed such that the quantization becomes uniform with respect to the human eye contrast sensitivity. More precisely, the HDR signal is perceptually transformed with SMPTE-2084 and uniformly quantized to 12 bits. This builds the input HDR signal X. For this image, the pixel distribution is computed and one TMO is computed per SDR rate value according to the optimization problem (21) in the paper [1]. Then, this image X is tone-mapped to produce the 8 bit base layer Y. The SDR image is encoded with the HEVC reference software HM 16.2 (18 ≤ QP ≤ 34) and inverse tone-mapped to obtain an HDR image XE.
The SDR reference signal is generated using the PTR operator proposed in [2], known to give good visual quality. Several images of different sizes and dynamic ranges have been tested with different constraint value D0. The results are shown for the input images AtriumNight, Nancy, FireEater and Tibul. These images are tone-mapped with the PTR [2] and shown on the following figure:

SDR reference images using Reinhard's TMO [2]
Not Found AtriumNight.png Not Found Nancy_cathedral.png Not Found FireEater.png Not Found Tibul.png
AtriumNight
Nancy_cathedral
FireEater
Tibul

For all these images, we test with the D0 constraint adjusted to 34.2dB (MSE=25). The second minimization aims to find the best rate allocation for the base and enhancement layers. The quality of the inverse tone mapped HDR image is improved by adding the enhancement layer Z, that is the residue represented on 12 bits. This residue is also encoded with the HM 16.2. Multiple encodings have been performed, with different combinations of base and enhancement QPs to test different rate allocations. For each value of the sum rate (RSDR + RHDR), the pair (QPSDR, QPHDR) leading to the best PSNR is retained, thus yielding the best R-D curve for each TMO. In the following section, we show results for the all of them.

Performance analysis of different TMO

In all the following figures, (a) plots the PSNR of the inverse tone mapped HDR image versus the SDR image bitrate, (b) plots the corresponding SDR PSNR with respect to the reference, and (c) plots the PSNR of the enhancement layer versus the total bitrate (SDR + HDR).

Image "AtriumNight"

R/D results for AtriumNight image
Not Found Results_AtriumNight.png

The proposed TMO gives the best R-D performance and the smallest distortion with respect to the SDR reference, especially at low bitrates. [17] and [18] solve a similar optimization problem, with however a simplified model of the variance of the residue, which leads to suboptimality and worst results in (a).
Concerning (c), the gap between R-D curves is reduced, compared to (a), where only the base layer is encoded. The enhancement layer helps the least efficient TMO to improve the HDR reconstruction, since we choose the best repartition for each TMO. However, the proposed TMO still keeps the best R-D performance although it respects a stricter constraint on the SDR perceptual quality, especially at low bitrates. The gain is around 0.1dB, for rates in the range (0.5, 1) bpp, compared to [17] or [18].

The following table shows all the SDR images obtained with each TMO at different QP. These images are comparable with the SDR image obtained with the Reinhard TMO


Comparison of SDR images obtained with the different TMOs
HEVC Base Layer QP = 18
[14]_AtriumNight_QP18
[17]_AtriumNight_QP18
[18]_AtriumNight_QP18
Proposed_AtriumNight_QP18
HEVC Base Layer QP = 26
[14]_AtriumNight_QP26
[17]_AtriumNight_QP26
[18]_AtriumNight_QP26
Proposed_AtriumNight_QP26
HEVC Base Layer QP = 34
[14]_AtriumNight_QP34
[17]_AtriumNight_QP34
[18]_AtriumNight_QP34
Proposed_AtriumNight_QP34

Image "Nancy_cathedral"

R/D results for Nancy image
Not Found Results_Nancy.png

The conclusions remains the same: the proposed TMO gives the best R-D performance and the smallest distortion with respect to the SDR reference.
Concerning (c), the gap between R-D curves is further reduced. In this case, [14] performs a bit better however this TMO is not constrained to a SDR reference. The proposed TMO still keeps good R-D performance although it respects a stricter constraint on the SDR perceptual quality.

The following table shows all the SDR images obtained with each TMO at different QP. These images are comparable with the SDR image obtained with the Reinhard TMO


Comparison of SDR images obtained with the different TMOs
HEVC Base Layer QP = 18
[14]_Nancy_cathedral_QP18
[17]_Nancy_cathedral_QP18
[18]_Nancy_cathedral_QP18
Proposed_Nancy_cathedral_QP18
HEVC Base Layer QP = 26
[14]_Nancy_cathedral_QP26
[17]_Nancy_cathedral_QP26
[18]_Nancy_cathedral_QP26
Proposed_Nancy_cathedral_QP26
HEVC Base Layer QP = 34
[14]_Nancy_cathedral_QP34
[17]_Nancy_cathedral_QP34
[18]_Nancy_cathedral_QP34
Proposed_Nancy_cathedral_QP34

Image "FireEater"

R/D results for FireEater image
Not Found Results_FireEaterD25.png

Interestingly, in this case, we observe that our optimization is easier to tune in order to meet the constraint on the SDR signal. Since [18] change the constrained optimization into an unconstrained optimization, where the tuning has to be done through the lagrangian multiplier, the distortion with the reference will depend on the image distribution. The proposed TMO does not give the best R-D performance but respect a much stricter constraint with respect to the SDR reference. For these reason we did not perform the second optimization on this case.

The following table shows all the SDR images obtained with each TMO at different QP. These images are comparable with the SDR image obtained with the Reinhard TMO


Comparison of SDR images obtained with the different TMOs
HEVC Base Layer QP = 18
[14]_FireEater_QP18
[17]_FireEater_QP18
[18]_FireEater_QP18
Proposed_FireEater_QP18
HEVC Base Layer QP = 26
[14]_FireEater_QP26
[17]_FireEater_QP26
[18]_FireEater_QP26
Proposed_FireEater_QP26
HEVC Base Layer QP = 34
[14]_FireEater_QP34
[17]_FireEater_QP34
[18]_FireEater_QP34
Proposed_FireEater_QP34

Here, we adjusted our constraint to 26dB to match the constraint from [18].


R/D results for FireEater image
Not Found Results_FireEaterD165.png

In this case, we observe that the proposed TMO still gives the best tradeoff between R-D performance and distortion with respect to the SDR reference.
Concerning (c), the gap between R-D curves is still very reduced. In this case too, [14] performs a bit better however this TMO is not constrained to a SDR reference. The proposed TMO still keeps good R-D performance although it respects a stricter constraint on the SDR perceptual quality.

The following table shows all the SDR images obtained with [17] and the proposed TMO (The only ones who change with the new constraint) at different QP. These images are comparable with the SDR image obtained with the Reinhard TMO


Comparison of SDR images obtained with the different TMOs
HEVC Base Layer QP = 18
HEVC Base Layer QP = 26
[17]_FireEater_QP18
Proposed_FireEater_QP18
[17]_FireEater_QP26
Proposed_FireEater_QP26
HEVC Base Layer QP = 34

Image "Tibul"

R/D results for Tibul image
Not Found Results_TibulD25.png

As for the FireEater image, we see that our optimization is easier to tune in order to meet the constraint on the SDR signal. The proposed TMO does not give the best R-D performance but respect a much stricter constraint with respect to the SDR reference. For these reason we did not perform the second optimization on this case.

The following table shows all the SDR images obtained with each TMO at different QP. These images are comparable with the SDR image obtained with the Reinhard TMO


Comparison of SDR images obtained with the different TMOs
HEVC Base Layer QP = 18
[14]_Tibul_QP18
[17]_Tibul_QP18
[18]_Tibul_QP18
Proposed_Tibul_QP18
HEVC Base Layer QP = 26
[14]_Tibul_QP26
[17]_Tibul_QP26
[18]_Tibul_QP26
Proposed_Tibul_QP26
HEVC Base Layer QP = 34
[14]_Tibul_QP34
[17]_Tibul_QP34
[18]_Tibul_QP34
Proposed_Tibul_QP34

Here, we adjusted our constraint to 29.5dB to match the constraint from [18].


R/D results for tibul image
Not Found Results_TibulD70.png

In this case, we observe that the proposed TMO still gives the best tradeoff between R-D performance and distortion withrespect to the SDR reference.
Concerning (c), the gap between R-D curves is still very reduced, however, the proposed TMO still respects a stricter constraint on the SDR perceptual quality, especially at low bitrates.

The following table shows all the SDR images obtained with [17] and the proposed TMO (The only ones who change with the new constraint) at different QP. These images are comparable with the SDR image obtained with the Reinhard TMO


Comparison of SDR images obtained with the different TMOs
HEVC Base Layer QP = 18
HEVC Base Layer QP = 26
[17]_Tibul_QP18
Proposed_Tibul_QP18
[17]_Tibul_QP26
Proposed_Tibul_QP26
HEVC Base Layer QP = 34

References