超分辨率图像重建

In our work:
We use patch-based sparse and redundant representation based prior, and We follow the work by Yang, Wright, Huang, and Ma [CVPR 2008, IEEE-TIP 2010], proposing an improved algorithm.
A min p q A q s.t. q pmin
L
Image Super-Resolution Using Sparse Representation By: Michael Elad
14
Training the Dictionaries:
Ah
p
k
h k
A hqk min Ah
min p A qk
qk k
2
s.t. qk
0
L
13
Image Super-Resolution Using Sparse Representation By: Michael Elad
Training the Dictionaries:
Ah
And then, the high-res. Patch is obtained by
Position k
Extraction of a patch from yh in location k is performed by
p R k yh
m
n
h k
Model Assumption: Every such patch can be represented sparsely over the dictionary Ah:
Interpolation
z
Lp p
h k
k 2

ph k A h qk
qk is ALSO the sparse representation of the low-resolution patch, with respect to the dictionary LAh.
Image Super-Resolution Using Sparse Representation By: Michael Elad
N=30
A
m=1000
For an image of size 1000× 1000 pixels, there are ~12,000 examples to train on
Given a low-res. Patch to be scaled-up, we start the resolution enhancement by sparse coding it, to find qk:
and now, lets go into the details …
Image Super-Resolution Using Sparse Representation By: Michael Elad
5
The Sparse-Land Prior [Aharon & Elad, ’06]
yh
n n
Interp. Q
Every patch from y should go through a process of resolution enhancement. The improved patches are then merged together into the final image (by averaging).
WGN v
z
+
z SHy h v
Our Task: Reverse the process – recover the high-resolution image from the low-resolution one
Image Super-Resolution Using Sparse Representation By: Michael Elad
Pre-process (high-res)

These will p be used for the pk dictionary training k
h k k
9
Image Super-Resolution Using Sparse Representation By: Michael Elad
p
h k k
Patch size: 9× 9
11
Pre-Processing Low-Res. Patches
Low-Res
x
Interpolation
We extract patches of size * [0 1 -1]T Dimensionality 9× 9 from each of Reduction these images, k and k k k * [-1 2 -1] By PCA concatenate them to form vector Patch size: ~30 h h one T * [-1 2 -1] of length 324 k k
2
Single Image Super-Resolution
z
Recovery Algorithm
?
ˆ yh
The reconstruction process should rely on:
The given low-resolution image The knowledge of S, H, and statistical properties of v, and Image behavior (prior).
Image Super-Resolution Using Sparse Representation By: Michael Elad
7
Low Versus High-Res. Patches
yhห้องสมุดไป่ตู้
Position k
y
n n
n n
We interpolate the low-res. Image in order to align the coordinate systems
Alternative: Bootstrapping
The given image to be scaled-up Simulate the degradation process
WGN v
z
Blur (H) and Decimation (S)
+
x
x SHzh v
Use this pair of images to generate the patches for the training
ˆh p k A hqk
resolution enhancement by sparse coding it, to find qk: 2 Remember h k hk k 2 k Ah k 0 2 qk k
Thus,Given Ah should be designed such that a low-res. Patch to be scaled-up, we start the
* [0 1 -1]
p p
p Bp
Image Super-Resolution Using Sparse Representation By: Michael Elad
12
Training the Dictionaries:
A

p
Remember
k k
K-SVD
40 iterations, L=3
Image Super-Resolution Using Sparse Representation By: Michael Elad
3
Core Idea (1) - Work on Patches
yh
y
We interpolate the low-res. Image in order to align the coordinate systems
Image Super-Resolution Using Sparse Representation By: Michael Elad
10
Pre-Processing High-Res. Patches
High-Res Low-Res Interpolation
x
z
Image Super-Resolution Using Sparse Representation By: Michael Elad
LA hqk p
A
k 2

8
Training the Dictionaries – General
Training pair(s)
We obtain a set of matching patch-pairs
p p k
h k k
Pre-process Interpolation (low-res)
* Joint work with Roman Zeyde and Matan Protter
Image Super-Resolution Using Sparse Representation By: Michael Elad
The Super-Resolution Problem
yh
Blur (H) and Decimation (S)
2
2
However, this approach disregards the fact that the resulting highres. patches are not used directly as the final result, but averaged due to overlaps between them. A better method (leading to better final scaled-up image) would be – Find Ah such that the following error is minimized:
yh
Position k
y
n n
n n
Interpolation Q
z
p
h k
?
p
k
h k k 2
z SHy h v
Lp p

L = blur + decimation + interpolation. This expression relates the low-res. patch to the high-res. one
?
We shall construct the high res. patch using the same sparse representation, imposed on a dictionary A h
We shall perform a sparse decomposition of the lowres. patch, w.r.t. a learned dictionary A
y Qz
Enhance Resolution
z
Image Super-Resolution Using Sparse Representation By: Michael Elad
4
Core Idea (2) – Learning [Yang et. al. ’08]
Enhance Resolution
p h k qk
such that
0
Ah
qk

ph k
6
ph qk k A h qk and
Image Super-Resolution Using Sparse Representation By: Michael Elad
n
Low Versus High-Res. Patches
Single Image Super-Resolution Using Sparse Representation *
Michael Elad
The Computer Science Department The Technion – Israel Institute of technology Haifa 32000, Israel
min y h ˆ yh
Ah
2 2
min y h y W R A hqk
Ah k T k
2
2
The constructed image
Image Super-Resolution Using Sparse Representation By: Michael Elad