Gradient-Based Tone Mapping for Rate-Distortion Optimized Backward-Compatible High Dynamic Range Compression

 
Image not found  Gradient-Based TMO for HDR Rate Distortion Optimization
This web page brings supplementary results/pictures/documents to the paper "Gradient-Based Tone Mapping for Rate-Distortion Optimized Backward-Compatible High Dynamic Range Compression" [1] submitted to IEEE Transactions on Image Processing.

Supplementary Documents:
Appendix 1 / Derivation of TMOs (37) and (38) in [1]
Appendix 2 / Derivation of DSDRREF (44) in [1]
TMO (32) Curves / With sample code to read and apply the TMOs

Contact:
D. GommeletA. RoumyC. GuillemotM. RopertJ. Le Tanou

We start with a slight reminder of the test process proposed in [1]. For more details informations, we refer you to the paper.
This paper addresses the problem of designing a global tone mapping operator for rate distortion optimized backward compatible compression of HDR images. We address the problem of tone mapping design for two different use cases leading to two different minimization problems. The first problem considered is the minimization of the distortion on the reconstructed HDR signal under a rate constraint on the SDR layer. The second problem remains the same minimization with an additional constraint to preserve a good quality for the SDR signal. Both the distortion and the rate are expressed as a function of the spatial gradient in the HDR images. Experiments show that the proposed rate and distortion models based on the HDR image gradient accurately predict the real image rate and distortion measures. Experimental results show that for the first minimization, the optimal rate-distortion performances are achieved, and that the second optimization yields the best tradeoff between rate-distortion performance and quality preservation of the SDR signal.

The following figure presents the HDR backward compatible encoding scheme:

Image not found.png

The original HDR image is first perceptually transformed using the SMPTE-2084 PQTF to follow the contrast sensitivity function of the human eye, and uniformly quantized to 12 bits. This provides the HDR content denoted X. X is then tone mapped either using the two proposed TMOs. The first TMO (32) in [1] minimize the HDR distortion of DHDR under a rate constraint RSDR and the second TMO (46) minimize the HDR distortion DHDR under a rate constraint RSDR and under a perceptual quality constraint for the SDR DSDRREF. The resulting SDR content is, then, encoded with the HEVC reference software(HM 16.2). The bitstream is send, decoded and can be directly displayed on legacy SDR displays or it can be inverse tone-mapped to adress HDR displays.

Image Test Set used for models validation

As explained above, we proposed new rate and distortion models based on the HDR image gradient. In [1], we made multiple tests to validate and demonstrate the accuracy of these models in comparison with state of the art models. All these tests has been carried out with 26 different HDR images and 3 different TMOs, where the TMO slopes are randomly drawn to yield a strictly increasing TMO that maps the whole range of the input HDR image. However, due to lack of space in the paper, we didn't show the used images. The following table shows all 26 images and their diversity:

SDR reference images using Reinhard's TMO [2]
Image not found Image not found Image not found Image not found
FireEater_001
FireEater_041
FireEater_111
FireEater_191
Image not found Image not found Image not found Image not found
Market_001
Market_241
Balloon_001
Balloon_201
Image not found Image not found Image not found Image not found
Tibul_001
Tibul_101
Tibul_151
Flamingo
Image not found Image not found Image not found Image not found
MackinacBridge
NiagaraFalls
Peppermill
playground_TMO
Image not found Image not found Image not found Image not found
Route66Museum
WaffleHouse
walkpath
doll
Image not found Image not found Image not found Image not found
AtriumNight
BigfogMap
LadyBirdRedwoods
memorial
Image not found Image not found
nancy_cathedral
oaks


Experimental Results using the first proposed TMO (32) in [1]

Minimization of DHDR under the constraint RSDR

In this section, we present the results of the proposed TMO (32) in [1] using different metrics. The TMO [21] and [23] are referenced, [2] and [3] respectively, at the bottom of this web page.

Image not found  R-D Results for image "AtriumNight"

ImageNotFound.png
R/D results using PSNR HDR for AtriumNight image
ImageNotFound.png
R/D results using tPSNR for AtriumNight image
ImageNotFound.png
R/D results using puPSNR for AtriumNight image
ImageNotFound.png
R/D results using puSSIM for AtriumNight image
ImageNotFound.png
R/D results using HDRVDP for AtriumNight image

Image not found  R-D Results for image "Balloon"

ImageNotFound.png
R/D results using PSNR HDR for Balloon image
ImageNotFound.png
R/D results using tPSNR for Balloon image
ImageNotFound.png
R/D results using puPSNR for Balloon image
ImageNotFound.png
R/D results using puSSIM for Balloon image
ImageNotFound.png
R/D results using HDRVDP for Balloon image

Image not found  R-D Results for image "Doll"

ImageNotFound.png
R/D results using PSNR HDR for Doll image
ImageNotFound.png
R/D results using tPSNR for Doll image
ImageNotFound.png
R/D results using puPSNR for Doll image
ImageNotFound.png
R/D results using puSSIM for Doll image
ImageNotFound.png
R/D results using HDRVDP for Doll image

Image not found  R-D Results for image "FireEater"

ImageNotFound.png
R/D results using PSNR HDR for FireEater image
ImageNotFound.png
R/D results using tPSNR for FireEater image
ImageNotFound.png
R/D results using puPSNR for FireEater image
ImageNotFound.png
R/D results using puSSIM for FireEater image
ImageNotFound.png
R/D results using HDRVDP for FireEater image

Image not found  R-D Results for image "Market"

ImageNotFound.png
R/D results using PSNR HDR for Market image
ImageNotFound.png
R/D results using tPSNR for Market image
ImageNotFound.png
R/D results using puPSNR for Market image
ImageNotFound.png
R/D results using puSSIM for Market image
ImageNotFound.png
R/D results using HDRVDP for Market image

Image not found  R-D Results for image "Memorial"

ImageNotFound.png
R/D results using PSNR HDR for Memorial image
ImageNotFound.png
R/D results using tPSNR for Memorial image
ImageNotFound.png
R/D results using puPSNR for Memorial image
ImageNotFound.png
R/D results using puSSIM for Memorial image
ImageNotFound.png
R/D results using HDRVDP for Memorial image

Image not found  R-D Results for image "Nancy_Cathedral"

ImageNotFound.png
R/D results using PSNR HDR for Nancy image
ImageNotFound.png
R/D results using tPSNR for Nancy image
ImageNotFound.png
R/D results using puPSNR for Nancy image
ImageNotFound.png
R/D results using puSSIM for Nancy image
ImageNotFound.png
R/D results using HDRVDP for Nancy image

Image not found  R-D Results for image "Oaks"

ImageNotFound.png
R/D results using PSNR HDR for Oaks image
ImageNotFound.png
R/D results using tPSNR for Oaks image
ImageNotFound.png
R/D results using puPSNR for Oaks image
ImageNotFound.png
R/D results using puSSIM for Oaks image
ImageNotFound.png
R/D results using HDRVDP for Oaks image

Image not found  R-D Results for image "Playground"

ImageNotFound.png
R/D results using PSNR HDR for Playground image
ImageNotFound.png
R/D results using tPSNR for Playground image
ImageNotFound.png
R/D results using puPSNR for Playground image
ImageNotFound.png
R/D results using puSSIM for Playground image
ImageNotFound.png
R/D results using HDRVDP for Playground image

Image not found  R-D Results for image "Tibul"

ImageNotFound.png
R/D results using PSNR HDR for Tibul image
ImageNotFound.png
R/D results using tPSNR for Tibul image
ImageNotFound.png
R/D results using puPSNR for Tibul image
ImageNotFound.png
R/D results using puSSIM for Tibul image
ImageNotFound.png
R/D results using HDRVDP for Tibul image

Image not found  R-D Results for image "Walkpath"

ImageNotFound.png
R/D results using PSNR HDR for Walkpath image
ImageNotFound.png
R/D results using tPSNR for Walkpath image
ImageNotFound.png
R/D results using puPSNR for Walkpath image
ImageNotFound.png
R/D results using puSSIM for Walkpath image
ImageNotFound.png
R/D results using HDRVDP for Walkpath image

Summary of R-D gains using PSNR HDR and comparison with Dynamic Range and Spatial Activity

  Playground Oaks Doll Nancy Tibul Memorial FireEater Market Balloon AtriumNight Walkpath
R-D Gain to [21] 4.8658% 6.9569% 4.4741% 29.105% 6.8340% 21.518% 26.853% 2.0402% 7.4479% 15.506% 6.2853%
R-D Gain to Linear TMO 4.8780% 1.6122% 1.6812% 6.5565% 3.2049% 4.3111% 11.354% 0.6156% 1.4390% 3.0996% 1.1762%
Spatial Activity 57.17 37.16 47.72 28.83 44.82 71.23 38.43 90.24 82.49 76.03 64.80
Dynamic Range 2.103 6.103 2.104 3.104 2.105 3.105 7.106 8.106 9.107 4.108 1.109

At high rates, the TMO always seems to converge towards the same R-D performances as [21] and towards the same R-D performances as the linear TMO at low rates.For each image, the R-D performances are different but the comportment seems to be the same regardless of the dynamic range or spatial activity. The "Dynamic Range" and "Spatial Activity" computation are defined in [3]



Experimental Results using the second proposed TMO (46) in [1]

Minimization of DHDR under the constraints RSDR and DSDRREF

In this section, we present the results of the proposed TMO (46) in [1] using different metrics. The TMO Reinhard, [25] and [26] are referenced, [7] [5] and [6] respectively, at the bottom of this web page.
The results presenting the "SDR quality constraint vs the SDR Rate" is only included in the PSNR HDR figures since it remains the same when changing the HDR metric.

Image not found  R-D Results with SDR Quality Constraint for image "AtriumNight"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for AtriumNight image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for AtriumNight image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for AtriumNight image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for AtriumNight image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for AtriumNight image

Image not found  R-D Results with SDR Quality Constraint for image "Balloon"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for Balloon image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for Balloon image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for Balloon image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for Balloon image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for Balloon image

Image not found  R-D Results with SDR Quality Constraint for image "Doll"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for Doll image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for Doll image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for Doll image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for Doll image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for Doll image

Image not found  R-D Results with SDR Quality Constraint for image "FireEater"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for FireEater image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for FireEater image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for FireEater image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for FireEater image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for FireEater image

Image not found  R-D Results with SDR Quality Constraint for image "Market"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for Market image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for Market image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for Market image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for Market image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for Market image

Image not found  R-D Results with SDR Quality Constraint for image "Memorial"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for Memorial image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for Memorial image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for Memorial image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for Memorial image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for Memorial image

Image not found  R-D Results with SDR Quality Constraint for image "Nancy_Cathedral"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for Nancy image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for Nancy image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for Nancy image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for Nancy image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for Nancy image

Image not found  R-D Results with SDR Quality Constraint for image "Oaks"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for Oaks image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for Oaks image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for Oaks image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for Oaks image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for Oaks image

Image not found  R-D Results with SDR Quality Constraint for image "Playground"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for Playground image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for Playground image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for Playground image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for Playground image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for Playground image

Image not found  R-D Results with SDR Quality Constraint for image "Tibul"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for Tibul image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for Tibul image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for Tibul image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for Tibul image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for Tibul image

Image not found  R-D Results with SDR Quality Constraint for image "Walkpath"

ImageNotFound.png
R/D results with SDR Quality Constraint using PSNR HDR for Walkpath image
ImageNotFound.png
R/D results with SDR Quality Constraint using tPSNR for Walkpath image
ImageNotFound.png
R/D results with SDR Quality Constraint using puPSNR for Walkpath image
ImageNotFound.png
R/D results with SDR Quality Constraint using puSSIM for Walkpath image
ImageNotFound.png
R/D results with SDR Quality Constraint using HDRVDP for Walkpath image



References