[AAAI 2020] Official repository of JSI-GAN.
This is the official repository of JSI-GAN (AAAI2020).
We provide the training and test code along with the trained weights and the dataset (train+test) used for JSI-GAN. If you find this repository useful, please consider citing our paper.
Reference:
Soo Ye Kim*, Jihyong Oh*, and Munchurl Kim, "JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video," AAAI Conference on Artificial Intelligence, 2020. (* equal contribution)
BibTeX
@inproceedings{kim2020jsigan,
title={JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video},
author={Kim, Soo Ye and Oh, Jihyong and Kim, Munchurl},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
year={2020}
}
Our code is implemented using Tensorflow, and was tested under the following setting:
<source_path>
.test
folder in <source_path>/data
.<source_path>/checkpoint_dir
.main.py
with the following options in parse_args:(i) For testing the .mat file input with scale factor 2:
python main.py --phase 'test_mat' --scale_factor 2 --test_data_path_LR_SDR './data/test/testset_SDR_x2.mat' --test_data_path_HR_HDR './data/test/testset_HDR.mat'
(ii) For testing the .mat file input with scale factor 4:
python main.py --phase 'test_mat' --scale_factor 4 --test_data_path_LR_SDR './data/test/testset_SDR_x4.mat' --test_data_path_HR_HDR './data/test/testset_HDR.mat'
(iii) For testing the .png file input with scale factor 2:
python main.py --phase 'test_png' --scale_factor 2 --test_data_path_LR_SDR './data/test/testset_SDR_x2.mat' --test_data_path_HR_HDR './data/test/testset_HDR.mat'
(iv) For testing the .mat file input with scale factor 4:
python main.py --phase 'test_png' --scale_factor 4 --test_data_path_LR_SDR './data/test/testset_SDR_x4.mat' --test_data_path_HR_HDR './data/test/testset_HDR.mat'
<source_path>/test_img_dir
. The YUV channels are saved separately as 16-bit PNG files.<source_path>/test_img_dir
. The YUV channels are saved separately as 16-bit PNG files.PNGtoYUV.m
Matlab code, which would save the .yuv video in the same location as the .png files. Please set the directory name accordingly. (Encoding the raw .yuv video to a compressed video format such as .mp4 can be done using ffmpeg)--test_data_path_LR_SDR
and --test_data_path_HR_HDR
.label
, GT
and PSNR
in the test_png function in net.py
.--test_patch
option defines the number of patches (H, W) to divide the input frame (e.g., (1, 1) means the full 4K frame will be entered, (2, 2) means that it will be divided into 2x2 2K frame patches and processed serially). You may modify this variable so that the testing works with your GPU.<source_path>
.train
folder in <source_path>/data
.main.py
with the following options in parse_args:(i) For training the model with scale factor 2:
python main.py --phase 'train' --scale_factor 2 --train_data_path_LR_SDR './data/train/SDR_youtube_80.mat' --train_data_path_HR_HDR './data/train/HDR_youtube_80.mat'
(ii) For training the model with scale factor 4:
python main.py --phase 'train' --scale_factor 4 --train_data_path_LR_SDR './data/train/SDR_youtube_80_x4.mat' --train_data_path_HR_HDR './data/train/HDR_youtube_80.mat'
<source_path>/checkpoint_dir
.--exp_num
option before training to another number than 1 to avoid overwriting the provided pre-trained weights so that the new weights are saved in a different folder (e.g. JSI-GAN_x2_exp2).<source_path>/logs
.Please contact us via email ([email protected] or [email protected]) for any problems regarding the released code.
The source code is free for research and education use only. Any commercial use should get formal permission first.