Understanding and evaluating blind deconvolution algorithms

Blind deconvolution
blurred image
?
kernel

sharp image
?
Rich literature, no perfect solution
Fergus et al. 06, Levin 06, Jia 07, Joshi et al. 08, Shan et al. 08
p(im y, kernel delta ) p(im x, kernel k * )
Claim 2- MAPk estimator succeeds:
p(k | y) is maximized by the true kernel k k *
Intuition: dimensionality asymmetry
Let x be an arbitrarily large image sampled from a sparse prior p ( x ) , and y k * x n
Then
Claim 1- MAPx,k estimator fails: The delta explanation is favored
- MAPx,k approach fails even with sparse prior
• Spatially uniform blur assumption is invalid • Compare blind deconvolution algorithms on the same dataset, Fergus et al. 06 significantly outperforms all alternatives Ground truth data available online
Understanding and evaluating blind deconvolution algorithms
Anat Levin1,2, Yair Weiss1,3,
Fredo Durand1, Bill Freeman1,4
Байду номын сангаас1MIT
CSAIL, 2Weizmann Institute, 3Hebrew University, 4Adobe
-|x|0.5
-|x|
-|x|0.25
x Derivative histogram from a natural image
x Parametric models
Derivative distributions in natural images are sparse:
log p( x) i xi ,
In this talk:
• No new algorithm
• What makes blind deconvolution hard? • Quantitatively evaluate recent algorithms on the same dataset
Blind deconvolution
Variation of dot traces at 4 corners
Note: opposite conclusion by Fergus et al., 2006
Summary
• Good estimator is more important than correct prior:
- MAPk approach can do deconvolution even with Gaussian prior
>
cheaper
P(blurred step edge)
| d 2 | 0.5 | d1 | 0.5
k=[0.5,0.5]
1 1
0.5
0.50.5 0.50.5 1.41
P(impulse)
d1 1
sum of derivatives:
<
P(blurred impulse)
d1 0.5
- Laplace approximation:
Brainard and Freeman 97, Bronstein et al. 05
Evaluation on 1D signals
Exact MAPk
Favor correct solution despite wrong prior!
Favors delta solution
MAPk estimation
MAPx,k- estimate x,k simultaneously
argmax
P( x , k| y P( k | y
)
MAPk- estimate k alone, marginalize x
argmax
) )dx
= P( x , k| y

Results in this paper:
blurred image y
sharp image x
kernel k
~105 measurements
Large, ~105 unknowns Small, ~102 unknowns
MAPx,k– Estimation unreliable. Number of measurements always lower than number of unknowns: #y<#x+#k MAPk – Estimation reliable. Many measurements for large images: #y>>#k
P(im y, kernel delta ) P(im x, kernel k * )
The MAPx,k failure
sharp blurred

i

?
i

The MAPx,k failure
15x15 windows 25x25 windows 45x45 windows
Approximate MAPk strategies
Marginalization on x is challenging to compute
p(k | y ) p( x, k | y )dx
Approximation strategies:
- Independence assumption in derivatives space: Levin NIPS06 - Variational approximation: Miskin and Mackay 00, Fergus et al. SIGGRAPH06
y kxn
blurred image Input (known) blur kernel sharp image noise Unknown, need to estimate
? ?

Natural image priors
Gaussian:
-x2
Log prob
Laplacian:
i
0.5
i
4.5
cheaper
Real image row: Noise and texture behave as impulses total derivative contrast reduced by blur
Why does MAPx,k fail?
• Too few measurements? Fails even with infinitely large image • Wrong prior? Fails even for signals sampled from the prior • Choice of estimator

1
Sparse priors in image processing
• Denoising
Simoncelli et al., Roth&Black • Inpainting Sapiro et al., Levin et al. • Super resolution Tappen et al. • Transparency Levin et al. • Demosaicing Tappen et al., Hel-Or et al. • Non blind deconvolution Levin et al.
1
Should favor sharper x explanations
The MAPx,k paradox
P(
,
Latent image
kernel
)>P(
,
Latent image
kernel
)
Claim 1: Let x be an arbitrarily large image sampled from a * sparse prior p ( x ) , and y k x n Then the delta explanation is favored
MAPx,k
MAPk, Gaussian prior
MAPk, variational approximation (Fergus et al.)
Ground truth data acquisition
4 images x 8 kernels = 32 test images
Data available online: http://www.wisdom.weizmann.ac.il/~levina/
Naïve MAPx,k estimation
Find a kernel k and latent image x minimizing:
1 2 log p( x, k | y) 2 | k x y | i xi ,