Initial commit.

README.md

@@ -0,0 +1,132 @@
+---
+license: apache-2.0
+tags:
+- super-image
+- image-super-resolution
+datasets:
+- div2k
+metrics:
+- pnsr
+- ssim
+---
+# Multi-scale Residual Network for Image Super-Resolution (MSRN)
+MSRN model pre-trained on DIV2K (800 images training, augmented to 4000 images, 100 images validation) for 2x, 3x and 4x image super resolution. It was introduced in the paper [Multi-scale Residual Network for Image Super-Resolution](https://openaccess.thecvf.com/content_ECCV_2018/html/Juncheng_Li_Multi-scale_Residual_Network_ECCV_2018_paper.html) by Li et al. (2018) and first released in [this repository](https://github.com/MIVRC/MSRN-PyTorch). 
+
+The goal of image super resolution is to restore a high resolution (HR) image from a single low resolution (LR) image. The image below shows the ground truth (HR), the bicubic upscaling x2 and model upscaling x2.
+
+![Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 4](images/msrn_4_4_compare.png "Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 4")
+## Model description
+The MSRN model proposes a feature extraction structure called the multi-scale residual block. This module can "adaptively detect image features at different scales" and "exploit the potential features of the image".
+## Intended uses & limitations
+You can use the pre-trained models for upscaling your images 2x, 3x and 4x. You can also use the trainer to train a model on your own dataset.
+### How to use
+The model can be used with the [super_image](https://github.com/eugenesiow/super-image) library:
+```bash
+pip install super-image
+```
+Here is how to use a pre-trained model to upscale your image:
+```python
+from super_image import MsrnModel, ImageLoader
+from PIL import Image
+import requests
+
+url = 'https://paperswithcode.com/media/datasets/Set5-0000002728-07a9793f_zA3bDjj.jpg'
+image = Image.open(requests.get(url, stream=True).raw)
+
+model = MsrnModel.from_pretrained('eugenesiow/msrn', scale=2)           # scale 2, 3 and 4 models available
+inputs = ImageLoader.load_image(image)
+preds = model(inputs)
+
+ImageLoader.save_image(preds, './scaled_2x.png')                        # save the output 2x scaled image to `./scaled_2x.png`
+ImageLoader.save_compare(inputs, preds, './scaled_2x_compare.png')      # save an output comparing the super-image with a bicubic scaling
+```
+## Training data
+The models for 2x, 3x and 4x image super resolution were pretrained on [DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K/), a dataset of 800 high-quality (2K resolution) images for training, augmented to 4000 images and uses a dev set of  100 validation images (images numbered 801 to 900). 
+## Training procedure
+### Preprocessing
+We follow the pre-processing and training method of [Wang et al.](https://arxiv.org/abs/2104.07566).
+Low Resolution (LR) images are created by using bicubic interpolation as the resizing method to reduce the size of the High Resolution (HR) images by x2, x3 and x4 times.
+During training, RGB patches with size of 64×64 from the LR input are used together with their corresponding HR patches. 
+Data augmentation is applied to the training set in the pre-processing stage where five images are created from the four corners and center of the original image. 
+
+The following code provides some helper functions to preprocess the data.
+```python
+from super_image.data import EvalDataset, TrainAugmentDataset, DatasetBuilder
+
+DatasetBuilder.prepare(
+    base_path='./DIV2K/DIV2K_train_HR',
+    output_path='./div2k_4x_train.h5',
+    scale=4,
+    do_augmentation=True
+)
+DatasetBuilder.prepare(
+    base_path='./DIV2K/DIV2K_val_HR',
+    output_path='./div2k_4x_val.h5',
+    scale=4,
+    do_augmentation=False
+)
+train_dataset = TrainAugmentDataset('./div2k_4x_train.h5', scale=4)
+val_dataset = EvalDataset('./div2k_4x_val.h5')
+```
+### Pretraining
+The model was trained on GPU. The training code is provided below:
+```python
+from super_image import Trainer, TrainingArguments, MsrnModel, MsrnConfig
+
+training_args = TrainingArguments(
+    output_dir='./results',                 # output directory
+    num_train_epochs=1000,                  # total number of training epochs
+)
+
+config = MsrnConfig(
+    scale=4,                                # train a model to upscale 4x
+)
+model = MsrnModel(config)
+
+trainer = Trainer(
+    model=model,                         # the instantiated model to be trained
+    args=training_args,                  # training arguments, defined above
+    train_dataset=train_dataset,         # training dataset
+    eval_dataset=val_dataset             # evaluation dataset
+)
+
+trainer.train()
+```
+## Evaluation results
+The evaluation metrics include [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio#Quality_estimation_with_PSNR) and [SSIM](https://en.wikipedia.org/wiki/Structural_similarity#Algorithm). 
+
+Evaluation datasets include:
+- Set5 - [Bevilacqua et al. (2012)](http://people.rennes.inria.fr/Aline.Roumy/results/SR_BMVC12.html)
+- Set14 - [Zeyde et al. (2010)](https://sites.google.com/site/romanzeyde/research-interests)
+- BSD100 - [Martin et al. (2001)](https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/)
+- Urban100 - [Huang et al. (2015)](https://sites.google.com/site/jbhuang0604/publications/struct_sr)
+
+The results columns below are represented below as `PSNR/SSIM`. They are compared against a Bicubic baseline.
+
+|Dataset  	    |Scale      |Bicubic  	        |msrn-bam  	                    |
+|---	        |---	    |---	            |---	                        |
+|Set5  	        |2x         |33.64/0.9292       |         |
+|Set5  	        |3x  	    |30.39/0.8678  	    |  	    |
+|Set5  	        |4x  	    |28.42/0.8101  	    |**32.19/0.8951**               |
+|Set14  	    |2x         |30.22/0.8683  	    |  	    |
+|Set14  	    |3x         |27.53/0.7737  	    |  	    |
+|Set14  	    |4x         |25.99/0.7023  	    |**28.67/0.7833**  	            |
+|BSD100  	    |2x  	    |29.55/0.8425  	    |  	    |
+|BSD100  	    |3x  	    |27.20/0.7382  	    |  	    |
+|BSD100  	    |4x  	    |25.96/0.6672  	    |**27.63/0.7374**  	            |
+|Urban100  	    |2x  	    |26.66/0.8408  	    |  	    |
+|Urban100  	    |3x  	    |  	                | 	    |
+|Urban100  	    |4x  	    |23.14/0.6573  	    |**26.12/0.7866**  	            |
+
+![Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2](images/msrn_2_4_compare.png "Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2")
+
+## BibTeX entry and citation info
+```bibtex
+@InProceedings{Li_2018_ECCV,
+    author = {Li, Juncheng and Fang, Faming and Mei, Kangfu and Zhang, Guixu},
+    title = {Multi-scale Residual Network for Image Super-Resolution},
+    booktitle = {The European Conference on Computer Vision (ECCV)},
+    month = {September},
+    year = {2018}
+}
+```

config.json

@@ -0,0 +1,8 @@
+{
+  "_name_or_path": "eugenesiow/msrn",
+  "data_parallel": false,
+  "model_type": "MSRN",
+  "n_feats": 64,
+  "n_blocks": 8,
+  "rgb_range": 255
+}

pytorch_model_4x.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:040478be5e5c0f8d3a12fcab7e3402715b2de9eec5fd39660886b41834bee4ec
+size 24343877