Supplementary Material: Deep Photo Enhancer: Unpaired ... · Supplementary Material: Deep Photo...

Supplementary Material:Deep Photo Enhancer:

Unpaired Learning for Image Enhancement fromPhotographs with GANs

Yu-Sheng Chen Yu-Ching Wang Man-Hsin Kao Yung-Yu Chuang

National Taiwan University

1 More comparisons on the MIT-Adobe 5K dataset

In the paper, we proposed three models for enhancing images, SL (supervisedlearning using the proposed generator trained on the MIT-Adobe 5K dataset),UL (the proposed 2-way GAN trained on the MIT-Adobe 5K dataset) and HDR(the proposed 2-way GAN trained on the collected HDR dataset). We com-pare the proposed models with five state-of-the-art methods, including Cycle-GAN [10], DPED [5], CLHE [8], NPEA [9] and FLLF [1]. Note that our modelcan be taken as an enhanced CycleGAN with three proposed improvements, abetter generator, a better WGAN model with the adaptive weighting schemeand a better 2-way GAN model with individual batch normalization layers. TheCycleGAN was trained on the collected HDR dataset. The DPED models aretailored with di↵erent mobile phones. Here, we show results of the DPED modelsfor iPhone6, iPhone7 and Nexus 5x.

This section compares these methods on some images from the MIT-Adobe-5K [2] testing dataset. In general, we have the following observations. Our modelstrained with the photographer’s labels approximate the labels reasonably well.DPED models vary a lot with di↵erent phone models. Though trained on thesame HDR dataset, CycleGAN cannot capture the characteristics of the datasetas well as our HDR model. It shows that the proposed improvements are e↵ectiveand important, at least for this application. CLHE, NPEA and FLLF are notrobust and could generate unnatural enhanced images at times. Our HDR modelcaptures the characteristics of the collected HDR dataset well and generates themost natural enhanced images.

2 Yu-Sheng Chen, Yu-Ching Wang, Man-Hsin Kao, and Yung-Yu Chuang

Input Label Our (SL) Our (UL)

Our (HDR) DPED (iPhone6) DPED (iPhone7) DPED (Nexus 5x)

CycleGAN (HDR) CLHE NPEA FLLF

Fig. 1: Comparisons of di↵erent methods on a MIT-adobe-5K testing image,a3552.

Supplementary Material: Deep Photo Enhancer 3




Fig. 2: Comparisons of di↵erent methods on a MIT-adobe-5K testing image,a0212


Input Label Our (HDR)

Our (SL) Our (UL) CycleGAN (HDR)

DPED (iPhone6) DPED (iPhone7) DPED (Nexus 5x)

CLHE NPEA FLLF






CLHE NPEA FLLF



2 More comparisons on images from the Internet

We show results of di↵erent methods on enhancing images collected from theInternet. We compare our HDR model with DPED and CLHE. For DPED, weshow the results using all three phone models. Although with good enhancementin general, the results of CLHE sometimes look unnatural, particularly on colors.The results of DPED vary among phone models, showing its dependence to phonemodels. In general, our results give natural results with enhanced color, contrastand details. Noe that the model was trained on HDR images. Thus, the resultsare ”HDR-like”. Sometime, it could look too prominent. It is however possibleto train a modest model with a set of images with that style.

We also provide an accompanying video showing the results of our HDRmodel on a video. Each frame is processed independently. However, there is noobvious temporal flick. It shows that our model is quite stable. The video alsodemonstrates that our model can be used for a wide variety of images.


Input Our (HDR) DPED (iPhone6)

DPED (iPhone7) DPED (Nexus 5x) CLHE

Fig. 8: Comparisons of di↵erent methods on an Internet image.


Input Our (HDR)

DPED (iPhone6) DPED (iPhone7)

DPED (Nexus 5x) CLHE



Input Our (HDR)





Input Our (HDR) DPED (iPhone6)

DPED (iPhone7) DPED (Nexus 5x) CLHE



Input Our (HDR)





3 The collected HDR dataset

We show sample images of the collected HDR dataset below.

Fig. 20: Sample images from the collected HDR dataset


4 Training

This section shows some figures about the training process. Figure 21 shows thetraining progress of di↵erent GAN models, the proposed A-WGAN, WGAN-GP, DRAGAN, LSGAN (local D), LSGAN and GAN. It can be seen that theproposed A-WGAN generates the best results while some GAN models couldtotally collapse. Figure 22 shows the discriminator loss along the training processfor di↵erent one-way GAN models trained on the MIT-Adobe 5K dataset. Forthe proposed model, the discriminator loss can be used as a good indicator forconvergence. Figure 23 shows the discriminator loss for the proposed two-wayGAN model with and without the individual batch normalization layers on theMIT-Adobe 5K and the collected HDR datasets. Although training on the MIT-Adobe 5K dataset is e↵ective without individual batch normalization, individualbatch normalization layers play an important role on the training with the HDRdataset.

0 50 100 150epoch

13

14

15

16

17

18

19

20

21

22

23

PSNR

Ours (A-WGAN)WGAN-GPDRAGANLSGAN(Local D)LSGANGAN

Fig. 21: PNSR values of testing on the MIT-Adobe-5K daatset with di↵erent one-way GAN architectures which use di↵erent GAN formulas, GAN [3], LSGAN [7],DRAGAN [6] and WGAN-GP [4]. (Local D: using the local discriminator pro-posed by CycleGAN [10].)


0 50 100 150epoch

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Disc

rimin

ator

loss

Ours (A-WGAN)WGAN-GPDRAGANLSGAN (Local D)LSGANGAN

Fig. 22: Discriminator loss of training on the MIT-Adobe-5K dataset for severalone-way GAN architectures, GAN [3], LSGAN [7], DRAGAN [6] and WGAN-GP [4]. The value can be used as a good indicator of convergence for our model.

0 20 40 60 80 100epoch

-50

0

50

100

150

200

250

300

Dis

crim

inat

or lo

ss

two-way trained on HDR (without iBN)two-way trained on HDR (with iBN)two-way trained on MIT-Adobe-5K (without iBN)two-way trained on MIT-Adobe-5K (with iBN)

Fig. 23: Discriminator loss of training on the MIT-Adobe-5K dataset and ourHDR dataset for the proposed two-way GAN architecture with and withoutindividual BN. It shows that individual BN is crucial for training on the HDRdataset.


References

1. Aubry, M., Paris, S., Hasino↵, S.W., Kautz, J., Durand, F.: Fast local laplacianfilters: Theory and applications. ACM Transactions on Graphics (TOG) 33(5), 167(2014)

2. Bychkovsky, V., Paris, S., Chan, E., Durand, F.: Learning photographic globaltonal adjustment with a database of input/output image pairs. In: IEEE Confer-ence on Computer Vision and Pattern Recognition (CVPR) (June 2011)

3. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair,S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in neuralinformation processing systems (NIPS). pp. 2672–2680 (2014)

4. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.: Improvedtraining of Wasserstein GANs. In: Advances in neural information processing sys-tems (NIPS) (2017)

5. Ignatov, A., Kobyshev, N., Vanhoey, K., Timofte, R., Van Gool, L.: DSLR-qualityphotos on mobile devices with deep convolutional networks. In: IEEE InternationalConference on Computer Vision (ICCV) (Oct 2017)

6. Kodali, N., Abernethy, J., Hays, J., Kira, Z.: On convergence and stability of GANs.In: arXiv preprint arXiv:1705.07215 (2017)

7. Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Paul Smolley, S.: Least squares gen-erative adversarial networks. In: The IEEE International Conference on ComputerVision (ICCV) (Oct 2017)

8. Wang, S., Cho, W., Jang, J., Abidi, M.A., Paik, J.: Contrast-dependent saturationadjustment for outdoor image enhancement. JOSA A 34(1), 2532–2542 (2017)

9. Wang, S., Zheng, J., Hu, H.M., Li, B.: Naturalness preserved enhancement algo-rithm for non-uniform illumination images. IEEE Transactions on Image Process-ing (TIP) 22(9), 3538–3548 (2013)

10. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translationusing cycle-consistent adversarial networks. In: The IEEE International Conferenceon Computer Vision (ICCV) (2017)

Supplementary Material: Deep Photo Enhancer: Unpaired ... · Supplementary Material: Deep Photo...

Documents

Transcript of Supplementary Material: Deep Photo Enhancer: Unpaired ... · Supplementary Material: Deep Photo...